-- Logs begin at Wed 2022-06-29 11:53:04 UTC, end at Wed 2022-06-29 12:58:12 UTC. -- Jun 29 12:19:18.811433 dv-cinder-812685-12-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 29 12:19:18.818651 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 29 12:19:18.820671 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jun 29 12:19:18 2022] *** Jun 29 12:19:18.820671 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 29 12:19:18.820671 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 29 12:19:18.820671 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: nodename: dv-cinder-812685-12-iscsi Jun 29 12:19:18.820671 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: machine: x86_64 Jun 29 12:19:18.820671 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: clock source: unix Jun 29 12:19:18.820671 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: pcre jit disabled Jun 29 12:19:18.821114 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: detected number of CPU cores: 4 Jun 29 12:19:18.821114 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: current working directory: / Jun 29 12:19:18.821114 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: detected binary path: /usr/bin/uwsgi-core Jun 29 12:19:18.821114 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: your processes number limit is 47680 Jun 29 12:19:18.821114 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: your memory page size is 4096 bytes Jun 29 12:19:18.821114 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: detected max file descriptor number: 2048 Jun 29 12:19:18.821114 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: lock engine: pthread robust mutexes Jun 29 12:19:18.821114 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: thunder lock: enabled Jun 29 12:19:18.821114 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 29 12:19:18.821114 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 29 12:19:18.866522 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: Python main interpreter initialized at 0x5559d33d69c0 Jun 29 12:19:18.866522 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: python threads support enabled Jun 29 12:19:18.866522 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: your server socket listen backlog is limited to 100 connections Jun 29 12:19:18.866522 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: your mercy for graceful operations on workers is 90 seconds Jun 29 12:19:18.866905 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: mapped 403077 bytes (393 KB) for 2 cores Jun 29 12:19:18.866991 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: *** Operational MODE: preforking *** Jun 29 12:19:18.867242 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: *** uWSGI is running in multiple interpreter mode *** Jun 29 12:19:18.867242 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: spawned uWSGI master process (pid: 80160) Jun 29 12:19:18.867441 dv-cinder-812685-12-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 29 12:19:18.868113 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: spawned uWSGI worker 1 (pid: 80169, cores: 1) Jun 29 12:19:18.868359 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: spawned uWSGI worker 2 (pid: 80170, cores: 1) Jun 29 12:19:18.874488 dv-cinder-812685-12-iscsi devstack@keystone.service[80160]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 29 12:19:24.322310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 29 12:19:24.322310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: from cryptography.utils import int_from_bytes Jun 29 12:19:24.323133 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 29 12:19:24.323133 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: from cryptography.utils import int_from_bytes Jun 29 12:19:24.323539 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 29 12:19:24.323539 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: from cryptography.utils import int_from_bytes Jun 29 12:19:24.323639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 29 12:19:24.323639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: from cryptography.utils import int_from_bytes Jun 29 12:19:24.372227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 29 12:19:24.372421 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 29 12:19:24.372617 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] command line args: [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 29 12:19:24.372891 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 29 12:19:24.373078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ================================================================================ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 29 12:19:24.373193 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 29 12:19:24.373320 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 29 12:19:24.373412 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] admin_token = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.373490 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] command line args: [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 29 12:19:24.373561 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] config_dir = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.373741 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.373829 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 29 12:19:24.373906 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ================================================================================ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 29 12:19:24.373982 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] config_source = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.374090 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] debug = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.374174 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] admin_token = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.374296 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.375546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config_dir = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.375546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.375658 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.375730 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config_source = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.375801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.375860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] debug = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.375917 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.375973 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.376060 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.376162 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.376162 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.376272 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.376272 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_config_append = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.376377 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.376439 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.376503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.376567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_dir = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.376630 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.376692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_file = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.376750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_config_append = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.376811 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.376875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.376934 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.376990 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_dir = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.376990 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.377089 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.377144 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.377207 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.377477 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.377565 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.377625 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.377696 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.377759 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.377827 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.377968 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.378086 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.378168 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.378240 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.378378 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.378524 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.378657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.378771 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.378903 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.378976 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.379100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.379174 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.379337 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.379452 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.379563 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.379680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.379803 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.380041 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.380162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.380282 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] publish_errors = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.380401 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.380521 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.380717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.380838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.380956 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.381076 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] publish_errors = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.381188 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.381299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.381415 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.381533 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.381662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.381782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] standard_threads = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.381900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.382013 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.382177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.382293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.382293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.382538 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] standard_threads = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.382660 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_journal = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.382780 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.382900 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_json = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.382900 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_stderr = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.383111 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.383222 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_syslog = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.383338 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.383338 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_journal = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.383558 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.383670 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_json = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.383781 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.383892 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_stderr = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.383892 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_syslog = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.384100 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.384222 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 29 12:19:24.384353 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.384469 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.384581 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.384692 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.384809 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.384929 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.385043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.385154 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.385271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.385389 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.385507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.385627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.385776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.385897 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.386014 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.386131 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.386243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.386366 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.386494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.386607 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.386728 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.386842 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.386952 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.387075 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.387220 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.387857 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.387986 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.388107 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.388229 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.388229 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.388527 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.388645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.388757 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.388757 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.388966 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.389089 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.389202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.389313 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.389425 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.389536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.389647 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.389767 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.389880 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.389992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.390109 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.390109 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.390319 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.390466 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.390588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.390701 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.390813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.390925 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.391045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.391163 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.391281 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.391399 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.391512 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.391623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.391742 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.391856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.391967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.392087 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.392199 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.392316 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.392436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.392549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.392666 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.392778 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.392895 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.393013 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.393133 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.393245 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.393357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.393468 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.393579 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.393690 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.393876 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.393991 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.394103 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.394220 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.394332 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.394467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.394597 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.394711 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.394822 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.394933 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.395051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.395169 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.395292 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.395404 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.395515 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.395880 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.395988 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.396087 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.396193 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.396288 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.396380 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.396472 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.396571 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.396671 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.396772 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.396867 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.396969 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.397081 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.397200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.397341 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.397465 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.397578 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.397690 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.397801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.397918 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.398031 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.398151 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.398255 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.398365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.398544 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.398657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.398772 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.398886 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.399007 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.399119 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.399240 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.399360 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.399474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.399587 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.399707 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.399821 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.399934 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.400053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.400167 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.400279 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.400392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.400512 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.400626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.400737 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.400850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.400968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.401089 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.401203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.401317 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.401419 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.401530 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.401642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.401762 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.401876 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.401988 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.402100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.402218 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.402332 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.402470 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.402586 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.402699 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.402825 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.402940 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.403053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.403174 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.403289 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.403402 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.403515 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.403627 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.403738 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.403858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.403972 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.403972 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.404189 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.404302 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.404416 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.404576 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.404700 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.404700 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.404911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.405023 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.405134 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.405325 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.405443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.405561 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.405676 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.405789 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.405901 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.406021 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.406143 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.406268 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.406381 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.406506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.406638 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.406782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.406932 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.407061 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.407176 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.407300 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.407414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.407527 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.407639 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.407639 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.407865 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.407865 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.408076 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.408076 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.408295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.408415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.408529 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.408642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.408754 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.408866 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.408989 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.external = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.408989 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.409200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.409200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.409410 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.409523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.external = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.409644 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.409783 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.409908 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.password = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.409908 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.token = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.410122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.410244 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.410358 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.410519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.410648 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.password = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.410771 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.410885 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.token = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.410998 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.411112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.411231 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.411354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.411474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.411589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.411702 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.411815 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.411918 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.412041 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.412156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.412270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.412270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.412481 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.412595 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.412716 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.412832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.412944 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.412944 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.413163 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.413278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.413402 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.413402 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.413616 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.413728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.413843 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.413956 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.414079 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.414201 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.414315 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.414442 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.414558 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.414671 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.414799 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.414915 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.415036 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.415150 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.415264 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.415376 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.415496 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.415611 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.415723 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.415723 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.415936 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.416049 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.416170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.416284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.416397 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.416510 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.416623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.416742 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.416865 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.416980 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.417092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.417092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.417303 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.417416 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.417537 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.417652 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.417765 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.417878 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.417992 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.418109 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.418234 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.418234 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.418491 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.418491 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.418708 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.418708 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.418996 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.419114 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.419228 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.419342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.419455 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.419580 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.419696 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.419816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.419930 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.420043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.420158 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.420279 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.420394 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.420508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.420622 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.420735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.420848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.420970 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.421085 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.421199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.421312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.421425 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.421538 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.421659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.421774 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.421887 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.422055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.422173 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.422288 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.422409 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.422542 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.422655 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.422769 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.422882 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.422995 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.423121 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.423308 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.423427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.423541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.423655 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.423768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.423890 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.424005 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.424118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.424231 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.424345 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.424495 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.424617 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.424738 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.424852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.424965 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.425078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.425191 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.425313 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.425435 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.425550 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.425663 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.425777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.425890 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.426017 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.426133 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.426246 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.426360 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.426522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.426637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.426763 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.426887 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.427002 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.427116 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.427229 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.427343 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.427465 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.427580 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.427700 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.427820 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.427937 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.428050 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.428172 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.428287 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.428400 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.428516 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.428629 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.428742 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.428863 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.428978 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.429091 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.429204 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.429317 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.429431 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.429553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.429675 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.429788 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.429902 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.430015 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.430129 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.430250 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.430365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.430492 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.430608 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.430721 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.430847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.430963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.431077 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.431190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.431304 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.431417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.431538 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.431654 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.431768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.431881 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.432001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.432115 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.432115 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.432338 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.432451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.432564 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.432676 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.432789 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.432910 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.433024 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.433291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.433405 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.433519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.433633 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.433757 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.433873 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.433873 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.434607 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.434725 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.434841 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.434968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.435084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.435084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.435280 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.435392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.435506 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.435628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.435628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.435843 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.435956 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.436076 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.436190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.436313 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.436429 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.436542 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.436656 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.436770 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.436884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.437006 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.437121 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.437234 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.437348 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.437461 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.437574 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.437696 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.437814 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.437929 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.438043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.438163 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.438277 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.438398 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.438569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.438684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.438797 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.438911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.439024 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.439149 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.439266 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.439380 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.439499 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.439613 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.439726 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.439848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.439963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.440077 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.440191 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.440304 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.440417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.440539 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.440653 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.440767 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.440880 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.440994 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.441108 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.441231 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.441346 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.441460 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.441573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.441686 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.441800 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] role.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.441987 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.442103 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] role.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.442219 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.442332 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.442473 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.442590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.442713 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.442828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.442942 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.443056 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.443169 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.443283 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.443404 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.443519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.443633 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.443746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.443859 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.443980 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.444095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.444210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.444324 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.444438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.444551 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.444673 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.444788 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.444901 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.445015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.445128 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.445242 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.445365 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.445480 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.445480 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.445694 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.445807 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.445920 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.446041 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.446157 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.446271 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.446385 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.446516 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.446630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.446630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.446858 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.446972 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.447086 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.447198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.447312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.447435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.447549 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.447665 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.447779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.447892 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.448005 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.448130 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.448246 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.448359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.448473 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.448586 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.448707 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.448823 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.448936 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.449049 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.449162 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.449275 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.449396 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.449512 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.449625 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.449769 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.449885 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.449998 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.450119 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.450235 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.450348 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.450475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.450591 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.450591 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.450816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.450932 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.451052 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.451167 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.451280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.451393 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.451515 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.451631 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.451744 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.451857 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.451971 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.452084 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.452205 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.452321 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.452434 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.452547 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.452667 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.452781 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.452906 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.453021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.453144 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.453258 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.453371 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.453484 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.453605 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.453721 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.453834 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.453947 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.454060 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.454173 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.454294 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.454445 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.454578 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.454695 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.454809 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.454923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.455037 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.455149 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.455271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.455385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.455507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.455630 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.455745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.455859 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.455973 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.456093 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.456209 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.456323 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.456436 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.456549 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.456662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.456784 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.456899 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.457013 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.457127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.457240 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.457354 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.457478 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.457595 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.457708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.457822 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.457936 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.458049 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.458171 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.458286 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.458400 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.458531 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.458645 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.458767 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 29 12:19:24.458891 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.458891 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.458891 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.459207 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 29 12:19:24.459207 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 29 12:19:24.488774 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.489002 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.489225 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.489310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.489515 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.489745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.498305 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.499087 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.527039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.527265 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.527912 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.528146 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.670187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.670364 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.719652 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.719797 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.719901 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.719980 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.784899 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.785069 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.785152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.785253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.785334 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.785440 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.785526 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.785657 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.785742 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.785873 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.785958 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.786088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.811845 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.812079 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.812304 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.812397 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.812526 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.812610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.812746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.812839 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.812963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.813047 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.813180 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.813261 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.813399 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.813617 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.824199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.824556 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.836975 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.837207 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.837309 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.837430 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.837514 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.837650 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.837732 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.837868 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.837952 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.838083 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.838165 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.838297 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.838379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.838568 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 29 12:19:24.855600 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.855810 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 29 12:19:24.856667 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.856811 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.856968 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 29 12:19:24.857113 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 29 12:19:24.857201 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.857326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.857409 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.857529 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 29 12:19:24.857621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.857799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.857882 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.858060 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.858318 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.858604 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.868597 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.868877 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.869144 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.869424 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.869519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.869740 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.870016 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.870299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.877851 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:24.878712 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:24.881148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:24.881429 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.881712 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.881992 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.882075 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:24.882270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.882350 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.882553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.882636 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.882824 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.882920 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.883096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.883179 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.883358 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.883434 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.883627 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.883717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.883895 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.883976 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.884161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.884435 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.884703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.911643 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.913264 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:24.913545 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.914168 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.915956 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:24.916355 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.919461 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:24.919749 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.920146 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.922993 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:24.923231 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.923633 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.927843 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:24.928129 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.930958 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:24.931247 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.932608 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:24.932897 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.933176 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.933457 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.933729 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.934693 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.934991 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.935263 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.936314 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:24.936604 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.936886 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.937169 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.937445 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.937718 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.937989 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.938257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.953750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.954027 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.956961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.957242 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.958782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:24.961929 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:24.962382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.962701 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.965640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.966077 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.967902 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:24.971259 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:24.971518 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:24.974909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:24.975179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.975480 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.975773 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.976062 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.976356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.976634 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.978573 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.978879 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.979167 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.979461 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.979752 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.980032 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.996085 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:24.996374 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.997254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:24.997521 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.000301 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:25.000589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.000860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.001128 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.005419 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.007179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.007462 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.009044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.010668 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:25.010785 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.010951 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.011065 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.011235 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.011518 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.011797 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.014363 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:25.014677 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.014966 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.015252 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.015545 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.025396 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:25.025677 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.025959 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.026242 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.026562 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.028818 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.029121 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.030135 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:25.030444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.030738 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.031028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.031313 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.032939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.033242 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.046895 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:25.050123 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:25.051266 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:25.053466 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.054567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:25.055242 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.056873 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:25.057152 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.057431 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.057901 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.060219 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.061875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:25.062162 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.062465 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.065552 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.067196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:25.069816 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.070435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.070732 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.071125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.071406 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.071487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:25.074745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.075049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.075518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.075810 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.080687 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:25.080975 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.081257 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.084899 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:25.085190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.085481 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.089073 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:25.089360 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.089643 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.089917 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.090190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.091112 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.091389 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.091672 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.091961 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.092235 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.092513 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.092776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.093043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.093318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.094007 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 29 12:19:25.094291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.094606 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.094888 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.095166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.095549 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.095842 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.096119 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.096402 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.096680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.096962 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.097231 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.097503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.097779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 29 12:19:25.131139 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5559d33d69c0 pid: 80169 (default app) Jun 29 12:19:25.134507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892da2e-bc6b-4438-8fc3-a1c8cfb206d2 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:25.134805 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892da2e-bc6b-4438-8fc3-a1c8cfb206d2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:25.135086 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892da2e-bc6b-4438-8fc3-a1c8cfb206d2 None None] PATH_INFO: `/v3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:25.136631 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5559d33d69c0 pid: 80170 (default app) Jun 29 12:19:25.139461 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 791 bytes} [Wed Jun 29 12:19:25 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:32.833483 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fc932-f9c1-4b1a-838c-a12f22956095 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:32.834046 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fc932-f9c1-4b1a-838c-a12f22956095 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:32.834046 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fc932-f9c1-4b1a-838c-a12f22956095 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:32.838328 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1/2] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:19:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:19:32.844404 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ef1b0-8f17-45e3-8f18-d07e541ca282 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:32.844795 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ef1b0-8f17-45e3-8f18-d07e541ca282 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:32.844981 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ef1b0-8f17-45e3-8f18-d07e541ca282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:32.916514 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: INFO dbcounter [None req-512ef1b0-8f17-45e3-8f18-d07e541ca282 None None] Registered counter for database keystone Jun 29 12:19:32.937645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG oslo_db.sqlalchemy.engines [None req-512ef1b0-8f17-45e3-8f18-d07e541ca282 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80169) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 29 12:19:32.943817 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writer thread running {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 29 12:19:33.128810 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG passlib.handlers.bcrypt [None req-512ef1b0-8f17-45e3-8f18-d07e541ca282 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80169) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 29 12:19:33.129147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG passlib.handlers.bcrypt [None req-512ef1b0-8f17-45e3-8f18-d07e541ca282 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80169) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 29 12:19:33.183660 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-512ef1b0-8f17-45e3-8f18-d07e541ca282 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:19:33.185315 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-512ef1b0-8f17-45e3-8f18-d07e541ca282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:19:33.305947 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 2/3] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:19:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 29 12:19:33.414888 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35fc932-f9c1-4b1a-838c-a12f22956095 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:33.485087 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: INFO dbcounter [None req-f35fc932-f9c1-4b1a-838c-a12f22956095 None None] Registered counter for database keystone Jun 29 12:19:33.504372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG oslo_db.sqlalchemy.engines [None req-f35fc932-f9c1-4b1a-838c-a12f22956095 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80170) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 29 12:19:33.510550 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writer thread running {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 29 12:19:33.628984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35fc932-f9c1-4b1a-838c-a12f22956095 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:33.633234 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630d559-7938-48aa-8b42-16fdc8bed059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:33.633901 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630d559-7938-48aa-8b42-16fdc8bed059 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:33.634192 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630d559-7938-48aa-8b42-16fdc8bed059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:33.634502 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630d559-7938-48aa-8b42-16fdc8bed059 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:33.642685 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f630d559-7938-48aa-8b42-16fdc8bed059 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:33.694520 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f630d559-7938-48aa-8b42-16fdc8bed059 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 29 12:19:33.694813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f630d559-7938-48aa-8b42-16fdc8bed059 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:33.704793 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 2/4] 10.222.0.52 () {58 vars in 1324 bytes} [Wed Jun 29 12:19:33 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 12:19:33.710830 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ef1b0-8f17-45e3-8f18-d07e541ca282 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:33.725122 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ef1b0-8f17-45e3-8f18-d07e541ca282 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:33.729505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491add33-d8ac-4c21-b273-9d36298d8434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:33.730636 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491add33-d8ac-4c21-b273-9d36298d8434 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:33.731448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491add33-d8ac-4c21-b273-9d36298d8434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:33.731733 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491add33-d8ac-4c21-b273-9d36298d8434 admin admin] PATH_INFO: `/v3/projects/a19795d69941402bbd95f8c9172fb339` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:33.735845 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491add33-d8ac-4c21-b273-9d36298d8434 admin admin] RBAC: Authorizing `identity:get_project(project_id=a19795d69941402bbd95f8c9172fb339)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:33.787097 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: WARNING keystone.common.rbac_enforcer.enforcer [None req-491add33-d8ac-4c21-b273-9d36298d8434 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 29 12:19:33.787442 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491add33-d8ac-4c21-b273-9d36298d8434 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:33.791966 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 3/5] 10.222.0.52 () {56 vars in 1207 bytes} [Wed Jun 29 12:19:33 2022] GET /identity/v3/projects/a19795d69941402bbd95f8c9172fb339 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:19:35.807261 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c247aa-1e70-4bdd-9504-7a4b62e6be97 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:35.807765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c247aa-1e70-4bdd-9504-7a4b62e6be97 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:35.807867 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c247aa-1e70-4bdd-9504-7a4b62e6be97 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:35.812660 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 3/6] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:19:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:19:35.817993 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a27d1-4af3-401b-946a-23bd4f72799e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:35.818368 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a27d1-4af3-401b-946a-23bd4f72799e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:35.818614 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a27d1-4af3-401b-946a-23bd4f72799e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:35.837718 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ec2a27d1-4af3-401b-946a-23bd4f72799e None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:19:35.839297 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ec2a27d1-4af3-401b-946a-23bd4f72799e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:19:35.847340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 4/7] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:19:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 29 12:19:35.869428 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c247aa-1e70-4bdd-9504-7a4b62e6be97 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:35.881912 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c247aa-1e70-4bdd-9504-7a4b62e6be97 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:35.886020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0f12c-cfde-4ffe-9a71-03af378bd084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:35.886681 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0f12c-cfde-4ffe-9a71-03af378bd084 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:35.887507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0f12c-cfde-4ffe-9a71-03af378bd084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:35.887826 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0f12c-cfde-4ffe-9a71-03af378bd084 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:35.895575 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0f12c-cfde-4ffe-9a71-03af378bd084 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:35.897701 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0f12c-cfde-4ffe-9a71-03af378bd084 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:35.906392 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 4/8] 10.222.0.52 () {58 vars in 1324 bytes} [Wed Jun 29 12:19:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 29 12:19:35.911503 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a27d1-4af3-401b-946a-23bd4f72799e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:35.921430 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a27d1-4af3-401b-946a-23bd4f72799e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:35.925806 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c09cc5-5730-48f8-94f2-a1fd2730bf6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:35.926705 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c09cc5-5730-48f8-94f2-a1fd2730bf6a admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:35.926994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c09cc5-5730-48f8-94f2-a1fd2730bf6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:35.927273 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c09cc5-5730-48f8-94f2-a1fd2730bf6a admin admin] PATH_INFO: `/v3/users/96982c6fb7694d38b8d0a9278de02353` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:35.928665 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c09cc5-5730-48f8-94f2-a1fd2730bf6a admin admin] RBAC: Authorizing `identity:get_user(user_id=96982c6fb7694d38b8d0a9278de02353)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:35.930515 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c09cc5-5730-48f8-94f2-a1fd2730bf6a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:35.932072 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 5/9] 10.222.0.52 () {56 vars in 1198 bytes} [Wed Jun 29 12:19:35 2022] GET /identity/v3/users/96982c6fb7694d38b8d0a9278de02353 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:37.993647 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803fbaab-7a79-4123-9487-f8fb8d01ffd8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:37.994180 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803fbaab-7a79-4123-9487-f8fb8d01ffd8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:37.994180 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803fbaab-7a79-4123-9487-f8fb8d01ffd8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:37.995570 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 5/10] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:19:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:19:38.001258 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308562fe-eb36-40b8-8418-4910d2e5e8a6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:38.001682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308562fe-eb36-40b8-8418-4910d2e5e8a6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:38.001830 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308562fe-eb36-40b8-8418-4910d2e5e8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:38.018871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-308562fe-eb36-40b8-8418-4910d2e5e8a6 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:19:38.020056 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-308562fe-eb36-40b8-8418-4910d2e5e8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:19:38.057239 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daacbff-b822-4f58-b38a-675c468ff2aa None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:38.057535 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daacbff-b822-4f58-b38a-675c468ff2aa None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:38.057820 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daacbff-b822-4f58-b38a-675c468ff2aa None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:38.059838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 6/11] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:19:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:19:38.064785 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb285b3e-da51-406c-9ec4-d8206ed3123d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:38.065077 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb285b3e-da51-406c-9ec4-d8206ed3123d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:38.065363 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb285b3e-da51-406c-9ec4-d8206ed3123d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:38.100548 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 6/12] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:19:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:19:38.105253 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG passlib.handlers.bcrypt [None req-cb285b3e-da51-406c-9ec4-d8206ed3123d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80170) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 29 12:19:38.105744 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG passlib.handlers.bcrypt [None req-cb285b3e-da51-406c-9ec4-d8206ed3123d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80170) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 29 12:19:38.123943 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308562fe-eb36-40b8-8418-4910d2e5e8a6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:38.147530 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:38.152082 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cb285b3e-da51-406c-9ec4-d8206ed3123d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:19:38.153722 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cb285b3e-da51-406c-9ec4-d8206ed3123d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:19:38.229808 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308562fe-eb36-40b8-8418-4910d2e5e8a6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:38.254392 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 7/13] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:19:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:19:38.263650 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cea5f1-7d9f-45f1-ad06-ffadda26b66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:38.264443 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cea5f1-7d9f-45f1-ad06-ffadda26b66e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:38.264732 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cea5f1-7d9f-45f1-ad06-ffadda26b66e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:38.265017 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cea5f1-7d9f-45f1-ad06-ffadda26b66e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:38.276260 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cea5f1-7d9f-45f1-ad06-ffadda26b66e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:19:38.276398 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb285b3e-da51-406c-9ec4-d8206ed3123d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:38.287639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-13cea5f1-7d9f-45f1-ad06-ffadda26b66e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:19:38.287639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:19:38.287639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:19:38.287639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:19:38.287639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:19:38.287639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:19:38.287639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:19:38.287639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:19:38.287639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:19:38.287639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:19:38.287639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:19:38.287639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:19:38.287639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:19:38.287639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:19:38.287639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:19:38.287639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:19:38.288535 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:19:38.288535 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:19:38.288535 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:19:38.288535 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:19:38.288535 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:19:38.288535 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:19:38.288535 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:19:38.288535 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:19:38.288535 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:19:38.288535 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:19:38.288535 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:19:38.288535 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:19:38.288535 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:19:38.288535 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:19:38.288535 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:19:38.288535 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:19:38.289522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:19:38.289522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:19:38.289522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:19:38.289522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:19:38.289522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:19:38.289522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:19:38.289522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:19:38.289522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:19:38.289522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:19:38.289522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:19:38.289522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:19:38.289522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:19:38.289522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 7/14] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:19:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:19:38.295177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cea5f1-7d9f-45f1-ad06-ffadda26b66e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:38.379103 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:38.379826 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb285b3e-da51-406c-9ec4-d8206ed3123d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:38.382902 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:38.399471 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cea5f1-7d9f-45f1-ad06-ffadda26b66e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:38.418951 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de69c0e5-aab7-4ad1-a38a-63129784455b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:38.419847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69c0e5-aab7-4ad1-a38a-63129784455b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:38.420132 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69c0e5-aab7-4ad1-a38a-63129784455b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:38.420407 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69c0e5-aab7-4ad1-a38a-63129784455b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:38.432771 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d81b5b-5364-4f6a-b4b0-68f757dd0dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:38.433436 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d81b5b-5364-4f6a-b4b0-68f757dd0dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:38.433772 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d81b5b-5364-4f6a-b4b0-68f757dd0dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:38.434055 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d81b5b-5364-4f6a-b4b0-68f757dd0dc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:38.434844 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d81b5b-5364-4f6a-b4b0-68f757dd0dc0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:19:38.435153 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d81b5b-5364-4f6a-b4b0-68f757dd0dc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:38.436560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d81b5b-5364-4f6a-b4b0-68f757dd0dc0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:38.448055 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 8/15] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:19:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:19:38.454145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d81b5b-5364-4f6a-b4b0-68f757dd0dc0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:38.487915 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de69c0e5-aab7-4ad1-a38a-63129784455b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:38.489549 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de69c0e5-aab7-4ad1-a38a-63129784455b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:38.554144 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d81b5b-5364-4f6a-b4b0-68f757dd0dc0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:38.589295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070796fd-4c49-4b77-8834-eb79a437179a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:38.590774 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070796fd-4c49-4b77-8834-eb79a437179a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:38.591069 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070796fd-4c49-4b77-8834-eb79a437179a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:38.591348 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070796fd-4c49-4b77-8834-eb79a437179a None admin] PATH_INFO: `/v3/users/96982c6fb7694d38b8d0a9278de02353` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:38.592762 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070796fd-4c49-4b77-8834-eb79a437179a None admin] RBAC: Authorizing `identity:get_user(user_id=96982c6fb7694d38b8d0a9278de02353)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:38.594242 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070796fd-4c49-4b77-8834-eb79a437179a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:38.595904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 9/16] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:19:38 2022] GET /identity/v3/users/96982c6fb7694d38b8d0a9278de02353 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:19:38.600582 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 8/17] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:19:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 12:19:38.603360 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070796fd-4c49-4b77-8834-eb79a437179a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:38.604226 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de69c0e5-aab7-4ad1-a38a-63129784455b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:38.608244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:38.620410 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:38.710643 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070796fd-4c49-4b77-8834-eb79a437179a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:38.717071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de69c0e5-aab7-4ad1-a38a-63129784455b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:38.747796 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efad32d-39ef-4f53-a420-30b533c6d628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:38.748589 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efad32d-39ef-4f53-a420-30b533c6d628 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:38.748876 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efad32d-39ef-4f53-a420-30b533c6d628 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:38.749149 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efad32d-39ef-4f53-a420-30b533c6d628 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:38.750329 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efad32d-39ef-4f53-a420-30b533c6d628 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:38.751654 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8909a-ba09-4db5-a309-ca29883b66c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:38.751812 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efad32d-39ef-4f53-a420-30b533c6d628 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:38.752431 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8909a-ba09-4db5-a309-ca29883b66c5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:38.752867 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8909a-ba09-4db5-a309-ca29883b66c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:38.753294 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8909a-ba09-4db5-a309-ca29883b66c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:38.753914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 10/18] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:19:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:38.759502 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efad32d-39ef-4f53-a420-30b533c6d628 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:38.774364 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8909a-ba09-4db5-a309-ca29883b66c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:38.775872 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f8909a-ba09-4db5-a309-ca29883b66c5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:38.801473 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b9f8909a-ba09-4db5-a309-ca29883b66c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:19:38.801473 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:19:38.801473 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:19:38.801473 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:19:38.801473 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:19:38.801473 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:19:38.801473 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:19:38.801473 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:19:38.801473 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:19:38.801473 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:19:38.801473 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:19:38.801473 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:19:38.801473 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:19:38.801473 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:19:38.801473 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:19:38.802546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:19:38.802546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:19:38.802546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:19:38.802546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:19:38.802546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:19:38.802546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:19:38.802546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:19:38.802546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:19:38.802546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:19:38.802546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:19:38.802546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:19:38.802546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:19:38.802546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:19:38.802546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:19:38.802546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:19:38.802546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:19:38.802546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:19:38.803472 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:19:38.803472 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:19:38.803472 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:19:38.803472 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:19:38.805720 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 9/19] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:19:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:19:38.811511 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8909a-ba09-4db5-a309-ca29883b66c5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:38.852261 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:38.871790 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:38.873451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efad32d-39ef-4f53-a420-30b533c6d628 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:38.906786 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c65868-757a-4451-9d60-aec4b0dcc0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:38.907444 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c65868-757a-4451-9d60-aec4b0dcc0ee None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:38.907732 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c65868-757a-4451-9d60-aec4b0dcc0ee None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:38.908047 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c65868-757a-4451-9d60-aec4b0dcc0ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:38.908858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c65868-757a-4451-9d60-aec4b0dcc0ee None admin] RBAC: Adding query filter params (role.id=3a7f4bf556344407a6d9540c89d13641, scope.domain.id=default, user.id=96982c6fb7694d38b8d0a9278de02353) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:19:38.909181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c65868-757a-4451-9d60-aec4b0dcc0ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:38.910651 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c65868-757a-4451-9d60-aec4b0dcc0ee None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:38.916780 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f8909a-ba09-4db5-a309-ca29883b66c5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:38.917242 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 11/20] 10.222.0.52 () {56 vars in 1322 bytes} [Wed Jun 29 12:19:38 2022] GET /identity/v3/role_assignments?user.id=96982c6fb7694d38b8d0a9278de02353&scope.domain.id=default&role.id=3a7f4bf556344407a6d9540c89d13641 => generated 233 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:38.956184 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7786e20d-09c7-4c8b-b01c-8065acc3a22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:38.956862 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786e20d-09c7-4c8b-b01c-8065acc3a22a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:38.957149 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786e20d-09c7-4c8b-b01c-8065acc3a22a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:38.957428 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786e20d-09c7-4c8b-b01c-8065acc3a22a None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:38.958172 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7786e20d-09c7-4c8b-b01c-8065acc3a22a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:19:38.958514 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7786e20d-09c7-4c8b-b01c-8065acc3a22a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:38.959933 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7786e20d-09c7-4c8b-b01c-8065acc3a22a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:38.979697 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 10/21] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:19:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:19:41.169816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a076f-31e9-4e9b-94ac-e9732059f198 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:41.170329 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a076f-31e9-4e9b-94ac-e9732059f198 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:41.170454 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a076f-31e9-4e9b-94ac-e9732059f198 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:41.171939 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 12/22] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:19:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:19:41.177415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538867e1-3051-441f-868e-fbf57ef27eb8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:41.177812 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538867e1-3051-441f-868e-fbf57ef27eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:41.177982 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538867e1-3051-441f-868e-fbf57ef27eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:41.200102 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-538867e1-3051-441f-868e-fbf57ef27eb8 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:19:41.201350 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-538867e1-3051-441f-868e-fbf57ef27eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:19:41.277567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 11/23] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:19:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:19:41.319935 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a076f-31e9-4e9b-94ac-e9732059f198 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:41.448393 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a076f-31e9-4e9b-94ac-e9732059f198 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:41.497192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208feae-782d-4bbe-91fb-d3307a1f485d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:41.498006 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208feae-782d-4bbe-91fb-d3307a1f485d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:41.498292 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208feae-782d-4bbe-91fb-d3307a1f485d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:41.498597 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208feae-782d-4bbe-91fb-d3307a1f485d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:41.503195 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029d8bd-307c-4a87-88b3-fc27bd5625c8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:41.503739 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029d8bd-307c-4a87-88b3-fc27bd5625c8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:41.504239 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029d8bd-307c-4a87-88b3-fc27bd5625c8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:41.505828 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 12/24] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:19:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:19:41.510639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5208feae-782d-4bbe-91fb-d3307a1f485d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:19:41.519253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5208feae-782d-4bbe-91fb-d3307a1f485d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:19:41.519253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:19:41.519253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:19:41.519253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:19:41.519253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:19:41.519253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:19:41.519253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:19:41.519253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:19:41.519253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:19:41.519253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:19:41.519253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:19:41.519253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:19:41.519253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:19:41.519253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:19:41.519253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:19:41.519253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:19:41.520269 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:19:41.520269 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:19:41.520269 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:19:41.520269 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:19:41.520269 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:19:41.520269 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:19:41.520269 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:19:41.520269 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:19:41.520269 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:19:41.520269 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:19:41.520269 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:19:41.520269 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:19:41.520269 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:19:41.520269 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:19:41.520269 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:19:41.520269 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:19:41.521359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:19:41.521359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:19:41.521359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:19:41.521359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:19:41.521359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:19:41.521359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:19:41.521359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:19:41.521359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:19:41.521359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:19:41.521359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:19:41.521359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:19:41.521359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:19:41.521359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 13/25] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:19:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:19:41.528090 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029d8bd-307c-4a87-88b3-fc27bd5625c8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:41.529762 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1d569-25d8-4657-a683-b9eccf43499d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:41.530065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1d569-25d8-4657-a683-b9eccf43499d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:41.530358 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e1d569-25d8-4657-a683-b9eccf43499d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:41.538758 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:41.559320 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e7e1d569-25d8-4657-a683-b9eccf43499d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:19:41.561346 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e7e1d569-25d8-4657-a683-b9eccf43499d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:19:41.670771 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 14/26] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:19:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:19:41.679969 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:41.698205 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6731f3-13b5-4eab-b304-4fcd61bb0128 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:41.699235 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6731f3-13b5-4eab-b304-4fcd61bb0128 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:41.700517 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6731f3-13b5-4eab-b304-4fcd61bb0128 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:41.702135 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 15/27] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:19:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:19:41.706409 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6731f3-13b5-4eab-b304-4fcd61bb0128 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:41.758734 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029d8bd-307c-4a87-88b3-fc27bd5625c8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:41.807789 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb3e53b-127d-43f6-9e33-a1600a500c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:41.808593 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3e53b-127d-43f6-9e33-a1600a500c12 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:41.809013 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3e53b-127d-43f6-9e33-a1600a500c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:41.809586 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3e53b-127d-43f6-9e33-a1600a500c12 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:41.810019 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb3e53b-127d-43f6-9e33-a1600a500c12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:19:41.810330 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb3e53b-127d-43f6-9e33-a1600a500c12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:41.811770 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb3e53b-127d-43f6-9e33-a1600a500c12 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:41.824464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 13/28] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:19:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:41.825947 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a5f04-9a73-4976-b21c-1b368f64d664 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:41.826251 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a5f04-9a73-4976-b21c-1b368f64d664 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:41.826569 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a5f04-9a73-4976-b21c-1b368f64d664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:41.847592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4f1a5f04-9a73-4976-b21c-1b368f64d664 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:19:41.849348 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4f1a5f04-9a73-4976-b21c-1b368f64d664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:19:41.925306 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6731f3-13b5-4eab-b304-4fcd61bb0128 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:41.976571 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 14/29] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:19:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:19:41.979350 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a5f04-9a73-4976-b21c-1b368f64d664 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:41.999679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34a022-1dd9-4fc9-b468-534dbff31110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:42.000791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34a022-1dd9-4fc9-b468-534dbff31110 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:42.001389 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34a022-1dd9-4fc9-b468-534dbff31110 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:42.002981 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34a022-1dd9-4fc9-b468-534dbff31110 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:42.006025 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:42.022569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34a022-1dd9-4fc9-b468-534dbff31110 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:42.024085 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34a022-1dd9-4fc9-b468-534dbff31110 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:42.062587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6b34a022-1dd9-4fc9-b468-534dbff31110 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:19:42.062587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:19:42.062587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:19:42.062587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:19:42.062587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:19:42.062587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:19:42.062587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:19:42.062587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:19:42.062587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:19:42.062587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:19:42.062587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:19:42.062587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:19:42.062587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:19:42.062587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:19:42.062587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:19:42.063479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:19:42.063479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:19:42.063479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:19:42.063479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:19:42.063479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:19:42.063479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:19:42.063479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:19:42.063479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:19:42.063479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:19:42.063479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:19:42.063479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:19:42.063479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:19:42.063479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:19:42.063479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:19:42.063479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:19:42.063479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:19:42.063479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:19:42.064831 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:19:42.064831 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:19:42.064831 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:19:42.064831 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:19:42.064831 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 16/30] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:19:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:19:42.064831 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34a022-1dd9-4fc9-b468-534dbff31110 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:42.065742 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:42.209972 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a5f04-9a73-4976-b21c-1b368f64d664 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:42.236226 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34a022-1dd9-4fc9-b468-534dbff31110 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:42.255071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c9f1f-bdb6-40df-8d41-1f5eecbeb179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:42.256558 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c9f1f-bdb6-40df-8d41-1f5eecbeb179 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:42.256994 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c9f1f-bdb6-40df-8d41-1f5eecbeb179 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:42.257445 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c9f1f-bdb6-40df-8d41-1f5eecbeb179 None admin] PATH_INFO: `/v3/users/96982c6fb7694d38b8d0a9278de02353` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:42.259686 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c9f1f-bdb6-40df-8d41-1f5eecbeb179 None admin] RBAC: Authorizing `identity:get_user(user_id=96982c6fb7694d38b8d0a9278de02353)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:42.262526 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c9f1f-bdb6-40df-8d41-1f5eecbeb179 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:42.265438 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 15/31] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:19:41 2022] GET /identity/v3/users/96982c6fb7694d38b8d0a9278de02353 => generated 247 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:42.266354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c9f1f-bdb6-40df-8d41-1f5eecbeb179 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:42.279769 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cc3039-5699-4744-8c45-fea146051151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:42.280916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc3039-5699-4744-8c45-fea146051151 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:42.281393 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc3039-5699-4744-8c45-fea146051151 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:42.281957 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc3039-5699-4744-8c45-fea146051151 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:42.282945 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cc3039-5699-4744-8c45-fea146051151 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:42.284648 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cc3039-5699-4744-8c45-fea146051151 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:42.392123 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c9f1f-bdb6-40df-8d41-1f5eecbeb179 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:42.399922 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:42.429412 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f027e3-6009-4e5c-aa3a-e32bb3748c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:42.430473 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f027e3-6009-4e5c-aa3a-e32bb3748c63 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:42.430955 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f027e3-6009-4e5c-aa3a-e32bb3748c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:42.431439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f027e3-6009-4e5c-aa3a-e32bb3748c63 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:42.432361 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f027e3-6009-4e5c-aa3a-e32bb3748c63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:19:42.432470 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 17/32] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Jun 29 12:19:42 2022] POST /identity/v3/roles => generated 225 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:19:42.432987 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f027e3-6009-4e5c-aa3a-e32bb3748c63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:42.434621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38be63-dd34-46c7-9d1f-b5848a9da5d0 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:42.434916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38be63-dd34-46c7-9d1f-b5848a9da5d0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:42.435200 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee38be63-dd34-46c7-9d1f-b5848a9da5d0 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:42.435960 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f027e3-6009-4e5c-aa3a-e32bb3748c63 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:42.436490 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 18/33] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:19:42.437230 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38be63-dd34-46c7-9d1f-b5848a9da5d0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:42.450555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 16/34] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:19:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:42.452322 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d698f9-1c66-4a6a-9832-061b2ed3fc22 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:42.452827 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d698f9-1c66-4a6a-9832-061b2ed3fc22 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:42.453297 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d698f9-1c66-4a6a-9832-061b2ed3fc22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:42.473446 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-77d698f9-1c66-4a6a-9832-061b2ed3fc22 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:19:42.475462 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-77d698f9-1c66-4a6a-9832-061b2ed3fc22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:19:42.499704 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:42.551813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 17/35] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:19:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:19:42.553146 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d698f9-1c66-4a6a-9832-061b2ed3fc22 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:42.566793 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee38be63-dd34-46c7-9d1f-b5848a9da5d0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:42.605692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb83edc-adb7-4a83-9bbd-0af824b18b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:42.606887 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb83edc-adb7-4a83-9bbd-0af824b18b44 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:42.607322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb83edc-adb7-4a83-9bbd-0af824b18b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:42.607865 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb83edc-adb7-4a83-9bbd-0af824b18b44 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:42.609463 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb83edc-adb7-4a83-9bbd-0af824b18b44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:42.611311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb83edc-adb7-4a83-9bbd-0af824b18b44 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:42.613678 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 19/36] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:19:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:42.619037 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb83edc-adb7-4a83-9bbd-0af824b18b44 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:42.672400 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:42.678304 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d698f9-1c66-4a6a-9832-061b2ed3fc22 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:42.715207 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d405456-01ec-46fd-8b37-633dfd91df7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:42.716156 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d405456-01ec-46fd-8b37-633dfd91df7b None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:42.716582 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d405456-01ec-46fd-8b37-633dfd91df7b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:42.717001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d405456-01ec-46fd-8b37-633dfd91df7b None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:42.717936 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d405456-01ec-46fd-8b37-633dfd91df7b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:42.719722 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d405456-01ec-46fd-8b37-633dfd91df7b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:42.737474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb83edc-adb7-4a83-9bbd-0af824b18b44 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:42.753209 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:42.775927 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0382292-12d7-4fe5-b372-976b64be2766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:42.776610 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0382292-12d7-4fe5-b372-976b64be2766 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:42.776897 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0382292-12d7-4fe5-b372-976b64be2766 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:42.777182 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0382292-12d7-4fe5-b372-976b64be2766 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:42.778213 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0382292-12d7-4fe5-b372-976b64be2766 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:42.779670 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0382292-12d7-4fe5-b372-976b64be2766 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:42.798461 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 20/37] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Jun 29 12:19:42 2022] POST /identity/v3/roles => generated 219 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:19:42.799345 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0382292-12d7-4fe5-b372-976b64be2766 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:42.876826 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 18/38] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jun 29 12:19:42 2022] POST /identity/v3/projects => generated 301 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:19:42.878865 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40880b5a-c6a9-4ad0-9d05-284a021962c7 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:42.879372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40880b5a-c6a9-4ad0-9d05-284a021962c7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:42.880559 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40880b5a-c6a9-4ad0-9d05-284a021962c7 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:42.882235 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 19/39] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:19:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:19:42.889393 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97161f5c-3d51-4d5c-9432-1e30a1071c0d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:42.890331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97161f5c-3d51-4d5c-9432-1e30a1071c0d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:42.891247 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97161f5c-3d51-4d5c-9432-1e30a1071c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:42.930189 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-97161f5c-3d51-4d5c-9432-1e30a1071c0d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:19:42.934740 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-97161f5c-3d51-4d5c-9432-1e30a1071c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:19:42.964538 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0382292-12d7-4fe5-b372-976b64be2766 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:43.030450 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b27c4-123f-4ab8-9c76-6f0a91d0a738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:43.031288 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b27c4-123f-4ab8-9c76-6f0a91d0a738 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:43.031578 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b27c4-123f-4ab8-9c76-6f0a91d0a738 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:43.031865 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b27c4-123f-4ab8-9c76-6f0a91d0a738 None admin] PATH_INFO: `/v3/domains/default/users/96982c6fb7694d38b8d0a9278de02353/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:43.035492 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130b27c4-123f-4ab8-9c76-6f0a91d0a738 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=96982c6fb7694d38b8d0a9278de02353, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:43.037658 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130b27c4-123f-4ab8-9c76-6f0a91d0a738 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:43.059721 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 21/40] 10.222.0.52 () {58 vars in 1361 bytes} [Wed Jun 29 12:19:42 2022] PUT /identity/v3/domains/default/users/96982c6fb7694d38b8d0a9278de02353/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:19:43.062028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 20/41] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:19:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:19:43.086180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b27c4-123f-4ab8-9c76-6f0a91d0a738 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:43.100472 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3efded-0b8e-45ca-86da-d4205f9740fe None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:43.101236 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3efded-0b8e-45ca-86da-d4205f9740fe None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:43.101722 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3efded-0b8e-45ca-86da-d4205f9740fe None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:43.103810 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 21/42] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:19:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:19:43.111487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3ee89-51ac-44a6-82ff-b6c0b2738b89 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:43.111487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3ee89-51ac-44a6-82ff-b6c0b2738b89 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:43.111487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3ee89-51ac-44a6-82ff-b6c0b2738b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:43.140582 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ceb3ee89-51ac-44a6-82ff-b6c0b2738b89 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:19:43.141022 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ceb3ee89-51ac-44a6-82ff-b6c0b2738b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:19:43.141503 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:43.234221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b27c4-123f-4ab8-9c76-6f0a91d0a738 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:43.249233 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 22/43] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:19:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:19:43.272315 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3ee89-51ac-44a6-82ff-b6c0b2738b89 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:43.272477 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67c89a-9847-4354-a7fe-eed8057af4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:43.273236 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67c89a-9847-4354-a7fe-eed8057af4d8 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:43.273534 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67c89a-9847-4354-a7fe-eed8057af4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:43.273818 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67c89a-9847-4354-a7fe-eed8057af4d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:43.274832 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67c89a-9847-4354-a7fe-eed8057af4d8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:43.277083 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67c89a-9847-4354-a7fe-eed8057af4d8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:43.283374 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:43.294034 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 22/44] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Jun 29 12:19:43 2022] POST /identity/v3/roles => generated 223 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:19:43.397386 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3ee89-51ac-44a6-82ff-b6c0b2738b89 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:43.436764 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ad22d-09dd-4f42-9b8e-0607b394a698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:43.437578 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ad22d-09dd-4f42-9b8e-0607b394a698 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:43.437865 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ad22d-09dd-4f42-9b8e-0607b394a698 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:43.438147 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ad22d-09dd-4f42-9b8e-0607b394a698 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:43.439439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ad22d-09dd-4f42-9b8e-0607b394a698 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:43.441187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ad22d-09dd-4f42-9b8e-0607b394a698 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:43.447307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 23/45] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:19:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:19:43.448564 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67c89a-9847-4354-a7fe-eed8057af4d8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:43.653294 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:43.660577 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67c89a-9847-4354-a7fe-eed8057af4d8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:43.693997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a81acda-f07b-4bf2-b877-1c06e8640d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:43.694681 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81acda-f07b-4bf2-b877-1c06e8640d17 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:43.694971 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81acda-f07b-4bf2-b877-1c06e8640d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:43.695252 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a81acda-f07b-4bf2-b877-1c06e8640d17 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:43.695977 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a81acda-f07b-4bf2-b877-1c06e8640d17 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:43.697434 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a81acda-f07b-4bf2-b877-1c06e8640d17 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:43.723310 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 23/46] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jun 29 12:19:43 2022] POST /identity/v3/projects => generated 312 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:19:45.108300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182328d-1221-41b7-9cc3-f63922fb2d61 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:45.108824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182328d-1221-41b7-9cc3-f63922fb2d61 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:45.108909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182328d-1221-41b7-9cc3-f63922fb2d61 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:45.110220 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 24/47] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:19:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:19:45.115400 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a49e5c-fc82-409b-880f-4564be33f840 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:45.115789 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a49e5c-fc82-409b-880f-4564be33f840 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:45.115983 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a49e5c-fc82-409b-880f-4564be33f840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:45.133045 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-88a49e5c-fc82-409b-880f-4564be33f840 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:19:45.134592 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-88a49e5c-fc82-409b-880f-4564be33f840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:19:45.211744 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 24/48] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:19:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:19:45.239566 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182328d-1221-41b7-9cc3-f63922fb2d61 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:45.316150 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:45.339422 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182328d-1221-41b7-9cc3-f63922fb2d61 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:45.373770 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4fd75-c7c3-4770-8a62-e3d8e9525f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:45.374598 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4fd75-c7c3-4770-8a62-e3d8e9525f34 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:45.374887 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4fd75-c7c3-4770-8a62-e3d8e9525f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:45.375168 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4fd75-c7c3-4770-8a62-e3d8e9525f34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:45.386534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4fd75-c7c3-4770-8a62-e3d8e9525f34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:19:45.396174 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-83c4fd75-c7c3-4770-8a62-e3d8e9525f34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:19:45.396174 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:19:45.396174 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:19:45.396174 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:19:45.396174 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:19:45.396174 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:19:45.396174 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:19:45.396174 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:19:45.396174 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:19:45.396174 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:19:45.396174 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:19:45.396174 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:19:45.396174 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:19:45.396174 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:19:45.396174 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:19:45.396174 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:19:45.397188 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:19:45.397188 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:19:45.397188 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:19:45.397188 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:19:45.397188 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:19:45.397188 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:19:45.397188 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:19:45.397188 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:19:45.397188 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:19:45.397188 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:19:45.397188 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:19:45.397188 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:19:45.397188 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:19:45.397188 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:19:45.397188 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:19:45.397188 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:19:45.398047 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:19:45.398047 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:19:45.398047 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:19:45.398047 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:19:45.398047 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:19:45.398047 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:19:45.398047 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:19:45.398047 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:19:45.398047 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:19:45.398047 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:19:45.398047 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:19:45.398047 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:19:45.398047 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 25/49] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:19:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:19:45.403928 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a49e5c-fc82-409b-880f-4564be33f840 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:45.483626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:45.507611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a49e5c-fc82-409b-880f-4564be33f840 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:45.541587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c748c55-8e68-4c08-bdc9-7b31bd4ef4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:45.542257 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c748c55-8e68-4c08-bdc9-7b31bd4ef4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:45.542590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c748c55-8e68-4c08-bdc9-7b31bd4ef4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:45.542891 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c748c55-8e68-4c08-bdc9-7b31bd4ef4d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:45.543843 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c748c55-8e68-4c08-bdc9-7b31bd4ef4d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:19:45.544155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c748c55-8e68-4c08-bdc9-7b31bd4ef4d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:45.545550 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c748c55-8e68-4c08-bdc9-7b31bd4ef4d0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:45.553712 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 25/50] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:19:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:45.559984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4fd75-c7c3-4770-8a62-e3d8e9525f34 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:45.658556 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4fd75-c7c3-4770-8a62-e3d8e9525f34 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:45.692440 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b9df6-bd8a-4b36-b9ac-d0a45d168103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:45.693321 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b9df6-bd8a-4b36-b9ac-d0a45d168103 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:45.693612 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b9df6-bd8a-4b36-b9ac-d0a45d168103 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:45.693893 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b9df6-bd8a-4b36-b9ac-d0a45d168103 None admin] PATH_INFO: `/v3/users/96982c6fb7694d38b8d0a9278de02353` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:45.695105 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b9df6-bd8a-4b36-b9ac-d0a45d168103 None admin] RBAC: Authorizing `identity:get_user(user_id=96982c6fb7694d38b8d0a9278de02353)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:45.696574 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b9df6-bd8a-4b36-b9ac-d0a45d168103 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:45.698221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 26/51] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:19:45 2022] GET /identity/v3/users/96982c6fb7694d38b8d0a9278de02353 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:45.704165 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c748c55-8e68-4c08-bdc9-7b31bd4ef4d0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:45.789575 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26983dce-f57b-49a2-95c6-8de2fa2e45c5 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:45.789868 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26983dce-f57b-49a2-95c6-8de2fa2e45c5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:45.790160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26983dce-f57b-49a2-95c6-8de2fa2e45c5 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:45.791590 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 27/52] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:19:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:19:45.796634 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7096dea4-bd03-4355-90ee-c6807aadd3da None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:45.796924 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7096dea4-bd03-4355-90ee-c6807aadd3da None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:45.797206 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7096dea4-bd03-4355-90ee-c6807aadd3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:45.803089 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:45.803738 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c748c55-8e68-4c08-bdc9-7b31bd4ef4d0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:45.814147 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7096dea4-bd03-4355-90ee-c6807aadd3da None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:19:45.815361 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7096dea4-bd03-4355-90ee-c6807aadd3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:19:45.837495 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f2594-c08c-49ed-ad0f-98f880437167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:45.838293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f2594-c08c-49ed-ad0f-98f880437167 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:45.838617 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f2594-c08c-49ed-ad0f-98f880437167 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:45.838904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f2594-c08c-49ed-ad0f-98f880437167 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:45.840632 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f2594-c08c-49ed-ad0f-98f880437167 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:45.842095 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f2594-c08c-49ed-ad0f-98f880437167 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:45.843759 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 26/53] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:19:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:45.849366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f2594-c08c-49ed-ad0f-98f880437167 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:45.858177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:45.882157 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 28/54] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:19:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:19:45.906734 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7096dea4-bd03-4355-90ee-c6807aadd3da None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:45.951758 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f2594-c08c-49ed-ad0f-98f880437167 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:45.987951 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e6e6d-ef70-48e8-875a-a987f17e6a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:45.988627 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e6e6d-ef70-48e8-875a-a987f17e6a00 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:45.988918 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e6e6d-ef70-48e8-875a-a987f17e6a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:45.989201 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e6e6d-ef70-48e8-875a-a987f17e6a00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:45.990056 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e6e6d-ef70-48e8-875a-a987f17e6a00 None admin] RBAC: Adding query filter params (role.id=3a7f4bf556344407a6d9540c89d13641, scope.domain.id=default, user.id=96982c6fb7694d38b8d0a9278de02353) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:19:45.990376 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e6e6d-ef70-48e8-875a-a987f17e6a00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:45.991919 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e6e6d-ef70-48e8-875a-a987f17e6a00 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:45.998822 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 27/55] 10.222.0.52 () {56 vars in 1322 bytes} [Wed Jun 29 12:19:45 2022] GET /identity/v3/role_assignments?user.id=96982c6fb7694d38b8d0a9278de02353&scope.domain.id=default&role.id=3a7f4bf556344407a6d9540c89d13641 => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:19:46.117496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7096dea4-bd03-4355-90ee-c6807aadd3da None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:46.155207 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9306f44d-eba9-43e8-9059-6880a722dd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:46.156012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9306f44d-eba9-43e8-9059-6880a722dd5c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:46.156301 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9306f44d-eba9-43e8-9059-6880a722dd5c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:46.156575 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9306f44d-eba9-43e8-9059-6880a722dd5c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:46.157745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9306f44d-eba9-43e8-9059-6880a722dd5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:46.159233 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9306f44d-eba9-43e8-9059-6880a722dd5c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:46.161212 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 29/56] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:19:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:19:46.167244 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e6e6d-ef70-48e8-875a-a987f17e6a00 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:46.240805 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:46.272245 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e6e6d-ef70-48e8-875a-a987f17e6a00 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:46.304213 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c7da1d-e0a3-4d67-b599-8f2171d90997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:46.304877 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7da1d-e0a3-4d67-b599-8f2171d90997 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:46.305167 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7da1d-e0a3-4d67-b599-8f2171d90997 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:46.305452 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7da1d-e0a3-4d67-b599-8f2171d90997 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:46.306182 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c7da1d-e0a3-4d67-b599-8f2171d90997 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:46.307677 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c7da1d-e0a3-4d67-b599-8f2171d90997 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:46.332841 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 28/57] 10.222.0.52 () {60 vars in 1140 bytes} [Wed Jun 29 12:19:46 2022] POST /identity/v3/projects => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:19:48.349376 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5b00e-4a57-428c-ab94-fbea2abddd9b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:48.349901 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5b00e-4a57-428c-ab94-fbea2abddd9b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:48.349984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5b00e-4a57-428c-ab94-fbea2abddd9b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:48.351347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 30/58] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:19:48.356456 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885918a7-7ab2-46f0-8231-0c6d80a3b0a0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:48.356862 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885918a7-7ab2-46f0-8231-0c6d80a3b0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:48.357027 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885918a7-7ab2-46f0-8231-0c6d80a3b0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:48.373841 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-885918a7-7ab2-46f0-8231-0c6d80a3b0a0 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:19:48.375073 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-885918a7-7ab2-46f0-8231-0c6d80a3b0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:19:48.439916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 29/59] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:19:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:19:48.464851 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5b00e-4a57-428c-ab94-fbea2abddd9b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:48.467803 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:48.565955 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5b00e-4a57-428c-ab94-fbea2abddd9b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:48.599527 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4b0fb-f158-4608-86d6-1403f74091e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:48.600255 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4b0fb-f158-4608-86d6-1403f74091e5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:48.600548 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4b0fb-f158-4608-86d6-1403f74091e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:48.600831 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4b0fb-f158-4608-86d6-1403f74091e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:48.602021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4b0fb-f158-4608-86d6-1403f74091e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:48.603497 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4b0fb-f158-4608-86d6-1403f74091e5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:48.605119 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 31/60] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:19:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:48.610690 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885918a7-7ab2-46f0-8231-0c6d80a3b0a0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:48.661715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:48.707816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885918a7-7ab2-46f0-8231-0c6d80a3b0a0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:48.739927 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b404791f-563e-4e95-a16b-0dfaf5f8c68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:48.740589 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404791f-563e-4e95-a16b-0dfaf5f8c68f None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:48.740875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404791f-563e-4e95-a16b-0dfaf5f8c68f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:48.741156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404791f-563e-4e95-a16b-0dfaf5f8c68f None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:48.741869 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b404791f-563e-4e95-a16b-0dfaf5f8c68f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:48.743336 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b404791f-563e-4e95-a16b-0dfaf5f8c68f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:48.767792 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 30/61] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 12:19:48 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:19:51.815354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fce7b5-e36e-4a47-9df7-74521f87f4e5 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:51.816392 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fce7b5-e36e-4a47-9df7-74521f87f4e5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:51.816883 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fce7b5-e36e-4a47-9df7-74521f87f4e5 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:51.818487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 32/62] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:19:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:19:51.823857 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d62955-255f-479b-8d6a-33eacc040bb1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:51.824866 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d62955-255f-479b-8d6a-33eacc040bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:51.825039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d62955-255f-479b-8d6a-33eacc040bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:51.847602 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b1d62955-255f-479b-8d6a-33eacc040bb1 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:19:51.848846 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b1d62955-255f-479b-8d6a-33eacc040bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:19:51.937396 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 31/63] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:19:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:19:51.975926 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fce7b5-e36e-4a47-9df7-74521f87f4e5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:52.082661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:52.094289 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f3806-0acd-477e-8c1d-0849357b3b01 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:52.094962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f3806-0acd-477e-8c1d-0849357b3b01 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:52.095429 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f3806-0acd-477e-8c1d-0849357b3b01 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:52.097023 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 32/64] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:19:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:19:52.107250 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d0d7a8-257a-4970-a0ac-efb9cc781568 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:52.107829 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d0d7a8-257a-4970-a0ac-efb9cc781568 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:52.108315 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d0d7a8-257a-4970-a0ac-efb9cc781568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:52.117432 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fce7b5-e36e-4a47-9df7-74521f87f4e5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:52.147591 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-65d0d7a8-257a-4970-a0ac-efb9cc781568 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:19:52.148796 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-65d0d7a8-257a-4970-a0ac-efb9cc781568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:19:52.154812 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdbe641-139f-40bd-8cd3-2ffcec556400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:52.155859 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdbe641-139f-40bd-8cd3-2ffcec556400 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:52.156364 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdbe641-139f-40bd-8cd3-2ffcec556400 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:52.156837 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdbe641-139f-40bd-8cd3-2ffcec556400 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:52.167482 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdbe641-139f-40bd-8cd3-2ffcec556400 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:19:52.176379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1cdbe641-139f-40bd-8cd3-2ffcec556400 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:19:52.176379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:19:52.176379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:19:52.176379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:19:52.176379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:19:52.176379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:19:52.176379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:19:52.176379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:19:52.176379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:19:52.176379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:19:52.176379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:19:52.176379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:19:52.176379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:19:52.176379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:19:52.176379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:19:52.176379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:19:52.177221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:19:52.177221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:19:52.177221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:19:52.177221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:19:52.177221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:19:52.177221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:19:52.177221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:19:52.177221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:19:52.177221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:19:52.177221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:19:52.177221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:19:52.177221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:19:52.177221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:19:52.177221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:19:52.177221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:19:52.177221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:19:52.177967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:19:52.177967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:19:52.177967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:19:52.177967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:19:52.177967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:19:52.177967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:19:52.177967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:19:52.177967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:19:52.177967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:19:52.177967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:19:52.177967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:19:52.177967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:19:52.180642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 33/65] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:19:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:19:52.186769 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdbe641-139f-40bd-8cd3-2ffcec556400 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:52.213235 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:52.283705 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 33/66] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:19:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:19:52.306577 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d0d7a8-257a-4970-a0ac-efb9cc781568 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:52.354707 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdbe641-139f-40bd-8cd3-2ffcec556400 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:52.390796 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:52.395037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a498c4a5-8e42-4296-a324-9170657f79a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:52.396208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498c4a5-8e42-4296-a324-9170657f79a1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:52.396718 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498c4a5-8e42-4296-a324-9170657f79a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:52.397198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a498c4a5-8e42-4296-a324-9170657f79a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:52.398154 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a498c4a5-8e42-4296-a324-9170657f79a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:19:52.398697 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a498c4a5-8e42-4296-a324-9170657f79a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:52.400317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a498c4a5-8e42-4296-a324-9170657f79a1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:52.410838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 34/67] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 12:19:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:52.412415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37ed98-8aff-4db6-ba6c-59487fac6235 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:52.412928 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37ed98-8aff-4db6-ba6c-59487fac6235 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:52.413407 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37ed98-8aff-4db6-ba6c-59487fac6235 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:52.413575 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d0d7a8-257a-4970-a0ac-efb9cc781568 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:52.415057 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 35/68] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:19:52.422490 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489af2e5-85b7-4645-98c2-234f655328f0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:52.422490 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489af2e5-85b7-4645-98c2-234f655328f0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:52.422490 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489af2e5-85b7-4645-98c2-234f655328f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:52.439140 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-489af2e5-85b7-4645-98c2-234f655328f0 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:19:52.440759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-489af2e5-85b7-4645-98c2-234f655328f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:19:52.448248 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd03a62-5b35-45de-a2ac-7bc18712ee25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:52.449596 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd03a62-5b35-45de-a2ac-7bc18712ee25 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:52.449885 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd03a62-5b35-45de-a2ac-7bc18712ee25 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:52.450225 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd03a62-5b35-45de-a2ac-7bc18712ee25 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:52.460414 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd03a62-5b35-45de-a2ac-7bc18712ee25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:19:52.468937 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9cd03a62-5b35-45de-a2ac-7bc18712ee25 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:19:52.468937 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:19:52.468937 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:19:52.468937 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:19:52.468937 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:19:52.468937 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:19:52.468937 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:19:52.468937 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:19:52.468937 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:19:52.468937 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:19:52.468937 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:19:52.468937 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:19:52.468937 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:19:52.468937 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:19:52.468937 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:19:52.468937 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:19:52.469958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:19:52.469958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:19:52.469958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:19:52.469958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:19:52.469958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:19:52.469958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:19:52.469958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:19:52.469958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:19:52.469958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:19:52.469958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:19:52.469958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:19:52.469958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:19:52.469958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:19:52.469958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:19:52.469958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:19:52.469958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:19:52.470932 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:19:52.470932 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:19:52.470932 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:19:52.470932 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:19:52.470932 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:19:52.470932 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:19:52.470932 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:19:52.470932 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:19:52.470932 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:19:52.470932 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:19:52.470932 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:19:52.470932 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:19:52.470932 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 34/69] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:19:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:19:52.475334 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd03a62-5b35-45de-a2ac-7bc18712ee25 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:52.505123 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:52.517552 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 36/70] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:19:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:19:52.519073 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780dc01-f9c8-4362-91da-5195777134bb None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:52.519442 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780dc01-f9c8-4362-91da-5195777134bb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:52.519799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780dc01-f9c8-4362-91da-5195777134bb None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:52.525616 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 37/71] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:19:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:19:52.526359 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780dc01-f9c8-4362-91da-5195777134bb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:52.591558 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd03a62-5b35-45de-a2ac-7bc18712ee25 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:52.624639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5db63-09b0-4285-88b6-b938f1fc7e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:52.625437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5db63-09b0-4285-88b6-b938f1fc7e19 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:52.625723 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5db63-09b0-4285-88b6-b938f1fc7e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:52.626002 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5db63-09b0-4285-88b6-b938f1fc7e19 None admin] PATH_INFO: `/v3/users/3f6a358f4bc044be94e44d1470a32c67` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:52.637412 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780dc01-f9c8-4362-91da-5195777134bb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:52.639867 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5db63-09b0-4285-88b6-b938f1fc7e19 None admin] RBAC: Authorizing `identity:get_user(user_id=3f6a358f4bc044be94e44d1470a32c67)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:52.641409 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5db63-09b0-4285-88b6-b938f1fc7e19 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:52.643127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 35/72] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:19:52 2022] GET /identity/v3/users/3f6a358f4bc044be94e44d1470a32c67 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:52.644481 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601433a-1fb4-42ac-9d13-df25477f3ccf None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:52.644717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601433a-1fb4-42ac-9d13-df25477f3ccf None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:52.645006 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a601433a-1fb4-42ac-9d13-df25477f3ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:52.661702 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:52.662139 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a601433a-1fb4-42ac-9d13-df25477f3ccf None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:19:52.663412 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a601433a-1fb4-42ac-9d13-df25477f3ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:19:52.674736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585fb100-7c0c-4f96-9582-1977ea6de3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:52.675417 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585fb100-7c0c-4f96-9582-1977ea6de3a5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:52.675709 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585fb100-7c0c-4f96-9582-1977ea6de3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:52.675992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585fb100-7c0c-4f96-9582-1977ea6de3a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:52.676751 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585fb100-7c0c-4f96-9582-1977ea6de3a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:19:52.677066 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585fb100-7c0c-4f96-9582-1977ea6de3a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:52.678489 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585fb100-7c0c-4f96-9582-1977ea6de3a5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:52.686929 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 38/73] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:19:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:52.687719 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585fb100-7c0c-4f96-9582-1977ea6de3a5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:52.729620 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 36/74] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:19:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:19:52.731075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0fa7a-0872-4e85-afa7-519746e81714 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:52.731369 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0fa7a-0872-4e85-afa7-519746e81714 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:52.731652 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0fa7a-0872-4e85-afa7-519746e81714 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:52.732914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 37/75] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:19:52.733618 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0fa7a-0872-4e85-afa7-519746e81714 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:52.756154 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:52.797845 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585fb100-7c0c-4f96-9582-1977ea6de3a5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:52.831800 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872cc3b1-bc44-4dc0-95dc-28dd021f7244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:52.832619 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cc3b1-bc44-4dc0-95dc-28dd021f7244 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:52.832924 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cc3b1-bc44-4dc0-95dc-28dd021f7244 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:52.833207 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cc3b1-bc44-4dc0-95dc-28dd021f7244 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:52.842156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0fa7a-0872-4e85-afa7-519746e81714 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:52.843694 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872cc3b1-bc44-4dc0-95dc-28dd021f7244 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:19:52.851896 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-872cc3b1-bc44-4dc0-95dc-28dd021f7244 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 12:19:52.851896 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:19:52.851896 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:19:52.851896 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:19:52.851896 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:19:52.851896 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:19:52.851896 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:19:52.851896 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:19:52.851896 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:19:52.851896 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:19:52.851896 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:19:52.851896 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:19:52.851896 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:19:52.851896 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:19:52.851896 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:19:52.851896 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:19:52.852823 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:19:52.852823 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:19:52.852823 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:19:52.852823 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:19:52.852823 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:19:52.852823 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:19:52.852823 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:19:52.852823 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:19:52.852823 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:19:52.852823 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:19:52.852823 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:19:52.852823 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:19:52.852823 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:19:52.852823 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:19:52.852823 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:19:52.852823 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:19:52.854001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:19:52.854001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:19:52.854001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:19:52.854001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:19:52.854001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:19:52.854001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:19:52.854001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:19:52.854001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:19:52.854001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:19:52.854001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:19:52.854001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 12:19:52.854001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:19:52.854001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 39/76] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 12:19:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:19:52.854001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872cc3b1-bc44-4dc0-95dc-28dd021f7244 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:52.880567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cbadc4-d701-48f1-880b-4b725c77bdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:52.881143 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cbadc4-d701-48f1-880b-4b725c77bdf8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:52.881425 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cbadc4-d701-48f1-880b-4b725c77bdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:52.881706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cbadc4-d701-48f1-880b-4b725c77bdf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:52.899134 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:52.947272 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cbadc4-d701-48f1-880b-4b725c77bdf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:52.948840 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cbadc4-d701-48f1-880b-4b725c77bdf8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:52.952429 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872cc3b1-bc44-4dc0-95dc-28dd021f7244 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:52.966027 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:52.987198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2781f-8db8-43e1-8cfd-c406e5405380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:52.988070 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2781f-8db8-43e1-8cfd-c406e5405380 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:52.988368 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2781f-8db8-43e1-8cfd-c406e5405380 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:52.988650 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2781f-8db8-43e1-8cfd-c406e5405380 None admin] PATH_INFO: `/v3/users/96982c6fb7694d38b8d0a9278de02353` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:52.989847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d2781f-8db8-43e1-8cfd-c406e5405380 None admin] RBAC: Authorizing `identity:get_user(user_id=96982c6fb7694d38b8d0a9278de02353)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:52.991313 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d2781f-8db8-43e1-8cfd-c406e5405380 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:52.992931 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 40/77] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:19:52 2022] GET /identity/v3/users/96982c6fb7694d38b8d0a9278de02353 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:52.994141 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33297fc-5e2d-47f7-906d-3c9bc7178f13 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:52.994460 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33297fc-5e2d-47f7-906d-3c9bc7178f13 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:52.994735 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33297fc-5e2d-47f7-906d-3c9bc7178f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:53.010982 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a33297fc-5e2d-47f7-906d-3c9bc7178f13 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:19:53.012715 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a33297fc-5e2d-47f7-906d-3c9bc7178f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:19:53.052838 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 38/78] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:19:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:19:53.053721 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cbadc4-d701-48f1-880b-4b725c77bdf8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:53.082235 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 41/79] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:19:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:19:53.087005 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33297fc-5e2d-47f7-906d-3c9bc7178f13 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:53.128964 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:53.156946 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cbadc4-d701-48f1-880b-4b725c77bdf8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:53.173314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:53.186401 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33297fc-5e2d-47f7-906d-3c9bc7178f13 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:53.191411 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7c62d-72bd-4ef2-a94c-a5493cb12bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:53.192893 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7c62d-72bd-4ef2-a94c-a5493cb12bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:53.193184 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7c62d-72bd-4ef2-a94c-a5493cb12bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:53.193474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7c62d-72bd-4ef2-a94c-a5493cb12bc3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:53.204321 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be7c62d-72bd-4ef2-a94c-a5493cb12bc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:19:53.212127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-0be7c62d-72bd-4ef2-a94c-a5493cb12bc3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:19:53.212127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:19:53.212127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:19:53.212127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:19:53.212127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:19:53.212127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:19:53.212127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:19:53.212127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:19:53.212127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:19:53.212127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:19:53.212127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:19:53.212127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:19:53.212127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:19:53.212127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:19:53.212127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:19:53.212127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:19:53.212957 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:19:53.212957 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:19:53.212957 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:19:53.212957 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:19:53.212957 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:19:53.212957 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:19:53.212957 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:19:53.212957 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:19:53.212957 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:19:53.212957 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:19:53.212957 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:19:53.212957 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:19:53.212957 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:19:53.212957 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:19:53.212957 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:19:53.212957 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:19:53.213712 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:19:53.213712 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:19:53.213712 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:19:53.213712 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:19:53.213712 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:19:53.213712 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:19:53.213712 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:19:53.213712 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:19:53.213712 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:19:53.213712 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:19:53.213712 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:19:53.213712 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:19:53.213712 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 39/80] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:19:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:19:53.214333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7c62d-72bd-4ef2-a94c-a5493cb12bc3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:53.224276 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b29a1-2ce1-4bbc-9dbd-e85aa4ff547c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:53.224956 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b29a1-2ce1-4bbc-9dbd-e85aa4ff547c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:53.225250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b29a1-2ce1-4bbc-9dbd-e85aa4ff547c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:53.225534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b29a1-2ce1-4bbc-9dbd-e85aa4ff547c None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:53.226290 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b29a1-2ce1-4bbc-9dbd-e85aa4ff547c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:19:53.226634 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b29a1-2ce1-4bbc-9dbd-e85aa4ff547c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:53.228034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b29a1-2ce1-4bbc-9dbd-e85aa4ff547c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:53.240517 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 42/81] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:19:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:53.240517 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b29a1-2ce1-4bbc-9dbd-e85aa4ff547c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:53.312365 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7c62d-72bd-4ef2-a94c-a5493cb12bc3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:53.335775 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b29a1-2ce1-4bbc-9dbd-e85aa4ff547c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:53.338020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:53.345996 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c541983-7b44-4081-afef-69b6fcd9e156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:53.346693 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c541983-7b44-4081-afef-69b6fcd9e156 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:53.346984 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c541983-7b44-4081-afef-69b6fcd9e156 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:53.347267 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c541983-7b44-4081-afef-69b6fcd9e156 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:53.370987 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d7f16-c248-43e8-bd07-eef2bb2c8575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:53.371937 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d7f16-c248-43e8-bd07-eef2bb2c8575 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:53.372234 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d7f16-c248-43e8-bd07-eef2bb2c8575 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:53.372520 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d7f16-c248-43e8-bd07-eef2bb2c8575 None admin] PATH_INFO: `/v3/projects/a19c893c70c84649bb5406d2cf9e2708` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:53.373749 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1d7f16-c248-43e8-bd07-eef2bb2c8575 None admin] RBAC: Authorizing `identity:get_project(project_id=a19c893c70c84649bb5406d2cf9e2708)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:53.375249 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1d7f16-c248-43e8-bd07-eef2bb2c8575 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:53.376966 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 43/82] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:19:53 2022] GET /identity/v3/projects/a19c893c70c84649bb5406d2cf9e2708 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:53.377736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d7f16-c248-43e8-bd07-eef2bb2c8575 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:53.397707 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:53.421113 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c541983-7b44-4081-afef-69b6fcd9e156 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:53.422576 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c541983-7b44-4081-afef-69b6fcd9e156 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:53.477525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d7f16-c248-43e8-bd07-eef2bb2c8575 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:53.510314 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962faa47-1177-4d37-898a-9cd89071139b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:53.511157 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962faa47-1177-4d37-898a-9cd89071139b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:53.511969 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962faa47-1177-4d37-898a-9cd89071139b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:53.512333 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962faa47-1177-4d37-898a-9cd89071139b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:53.513515 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962faa47-1177-4d37-898a-9cd89071139b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:53.515004 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962faa47-1177-4d37-898a-9cd89071139b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:53.521298 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 44/83] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:19:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:53.521298 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962faa47-1177-4d37-898a-9cd89071139b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:53.521519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 40/84] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:19:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:19:53.521519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c541983-7b44-4081-afef-69b6fcd9e156 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:53.563317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:53.597560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:53.620766 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c541983-7b44-4081-afef-69b6fcd9e156 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:53.622389 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962faa47-1177-4d37-898a-9cd89071139b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:53.656514 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2468a33-f76c-44c1-8be5-cbb55ee77e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:53.657195 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2468a33-f76c-44c1-8be5-cbb55ee77e03 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:53.657489 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2468a33-f76c-44c1-8be5-cbb55ee77e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:53.657776 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2468a33-f76c-44c1-8be5-cbb55ee77e03 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:53.658183 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be916d7-238e-40a3-8ea9-4eb1ec050348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:53.658567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2468a33-f76c-44c1-8be5-cbb55ee77e03 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:19:53.658881 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2468a33-f76c-44c1-8be5-cbb55ee77e03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:53.659102 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be916d7-238e-40a3-8ea9-4eb1ec050348 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:53.659395 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be916d7-238e-40a3-8ea9-4eb1ec050348 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:53.659682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be916d7-238e-40a3-8ea9-4eb1ec050348 None admin] PATH_INFO: `/v3/users/3f6a358f4bc044be94e44d1470a32c67` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:53.660269 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2468a33-f76c-44c1-8be5-cbb55ee77e03 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:53.660891 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be916d7-238e-40a3-8ea9-4eb1ec050348 None admin] RBAC: Authorizing `identity:get_user(user_id=3f6a358f4bc044be94e44d1470a32c67)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:53.662345 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be916d7-238e-40a3-8ea9-4eb1ec050348 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:53.664030 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 41/85] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:19:53 2022] GET /identity/v3/users/3f6a358f4bc044be94e44d1470a32c67 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:53.664803 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be916d7-238e-40a3-8ea9-4eb1ec050348 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:53.671997 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 45/86] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 12:19:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:53.672744 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2468a33-f76c-44c1-8be5-cbb55ee77e03 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:53.763294 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be916d7-238e-40a3-8ea9-4eb1ec050348 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:53.774784 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2468a33-f76c-44c1-8be5-cbb55ee77e03 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:53.776320 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:53.805507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296af71-c461-4375-a163-e6269634f545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:53.806176 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296af71-c461-4375-a163-e6269634f545 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:53.806501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296af71-c461-4375-a163-e6269634f545 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:53.806791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296af71-c461-4375-a163-e6269634f545 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:53.807633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3296af71-c461-4375-a163-e6269634f545 None admin] RBAC: Adding query filter params (role.id=3bf034b395b44d88b3af67da01a6ed82, scope.project.id=a19c893c70c84649bb5406d2cf9e2708, user.id=3f6a358f4bc044be94e44d1470a32c67) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:19:53.807959 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3296af71-c461-4375-a163-e6269634f545 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:53.809562 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3aeabf-409b-4d2c-a75d-483de4146684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:53.809933 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3296af71-c461-4375-a163-e6269634f545 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:53.810233 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3aeabf-409b-4d2c-a75d-483de4146684 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:53.810552 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3aeabf-409b-4d2c-a75d-483de4146684 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:53.810832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3aeabf-409b-4d2c-a75d-483de4146684 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:53.811653 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3aeabf-409b-4d2c-a75d-483de4146684 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:53.812958 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3aeabf-409b-4d2c-a75d-483de4146684 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:53.815361 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 42/87] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:19:53 2022] GET /identity/v3/role_assignments?user.id=3f6a358f4bc044be94e44d1470a32c67&scope.project.id=a19c893c70c84649bb5406d2cf9e2708&role.id=3bf034b395b44d88b3af67da01a6ed82 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:53.816146 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296af71-c461-4375-a163-e6269634f545 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:53.837168 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:53.843067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 46/88] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 12:19:53 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:19:53.844048 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3aeabf-409b-4d2c-a75d-483de4146684 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:53.960835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296af71-c461-4375-a163-e6269634f545 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:54.002480 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f2496-0088-4928-aee7-179bc2fac3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:54.003730 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f2496-0088-4928-aee7-179bc2fac3c5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:54.004222 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f2496-0088-4928-aee7-179bc2fac3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:54.004657 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f2496-0088-4928-aee7-179bc2fac3c5 None admin] PATH_INFO: `/v3/projects/a19c893c70c84649bb5406d2cf9e2708` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:54.006128 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f2496-0088-4928-aee7-179bc2fac3c5 None admin] RBAC: Authorizing `identity:get_project(project_id=a19c893c70c84649bb5406d2cf9e2708)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:54.006736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3aeabf-409b-4d2c-a75d-483de4146684 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:54.007983 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f2496-0088-4928-aee7-179bc2fac3c5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:54.009916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 43/89] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:19:53 2022] GET /identity/v3/projects/a19c893c70c84649bb5406d2cf9e2708 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:54.017734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f2496-0088-4928-aee7-179bc2fac3c5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:54.059359 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c39414-160b-41a6-8204-fbf3e97d3280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:54.060127 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c39414-160b-41a6-8204-fbf3e97d3280 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:54.060250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c39414-160b-41a6-8204-fbf3e97d3280 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:54.060529 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c39414-160b-41a6-8204-fbf3e97d3280 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:54.094143 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:54.135086 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:54.150031 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c39414-160b-41a6-8204-fbf3e97d3280 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:54.151851 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c39414-160b-41a6-8204-fbf3e97d3280 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:54.168529 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f2496-0088-4928-aee7-179bc2fac3c5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:54.217221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec1632-6dc4-46b8-9550-873b09b7f6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:54.218023 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec1632-6dc4-46b8-9550-873b09b7f6ae None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:54.218311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec1632-6dc4-46b8-9550-873b09b7f6ae None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:54.218624 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec1632-6dc4-46b8-9550-873b09b7f6ae None admin] PATH_INFO: `/v3/users/3f6a358f4bc044be94e44d1470a32c67` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:54.220776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec1632-6dc4-46b8-9550-873b09b7f6ae None admin] RBAC: Authorizing `identity:get_user(user_id=3f6a358f4bc044be94e44d1470a32c67)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:54.222368 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec1632-6dc4-46b8-9550-873b09b7f6ae None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:54.224209 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 44/90] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:19:54 2022] GET /identity/v3/users/3f6a358f4bc044be94e44d1470a32c67 => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:54.225034 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec1632-6dc4-46b8-9550-873b09b7f6ae None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:54.287187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 47/91] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:19:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:19:54.288066 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c39414-160b-41a6-8204-fbf3e97d3280 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:54.361377 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:54.361677 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec1632-6dc4-46b8-9550-873b09b7f6ae None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:54.407624 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467f0f2-7203-4f0f-b70f-bf957e8586cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:54.408627 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467f0f2-7203-4f0f-b70f-bf957e8586cd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:54.409114 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467f0f2-7203-4f0f-b70f-bf957e8586cd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:54.409597 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2467f0f2-7203-4f0f-b70f-bf957e8586cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:54.410709 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467f0f2-7203-4f0f-b70f-bf957e8586cd None admin] RBAC: Adding query filter params (role.id=3a7f4bf556344407a6d9540c89d13641, scope.project.id=a19c893c70c84649bb5406d2cf9e2708, user.id=96982c6fb7694d38b8d0a9278de02353) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:19:54.411254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467f0f2-7203-4f0f-b70f-bf957e8586cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:54.412892 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2467f0f2-7203-4f0f-b70f-bf957e8586cd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:54.418585 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c39414-160b-41a6-8204-fbf3e97d3280 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:54.419570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 45/92] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:19:54 2022] GET /identity/v3/role_assignments?user.id=96982c6fb7694d38b8d0a9278de02353&scope.project.id=a19c893c70c84649bb5406d2cf9e2708&role.id=3a7f4bf556344407a6d9540c89d13641 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:54.420572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467f0f2-7203-4f0f-b70f-bf957e8586cd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:54.442287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:54.488998 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6bf45-ab3e-44d5-8f72-be8d4bf939d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:54.490008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6bf45-ab3e-44d5-8f72-be8d4bf939d7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:54.490545 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6bf45-ab3e-44d5-8f72-be8d4bf939d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:54.491044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6bf45-ab3e-44d5-8f72-be8d4bf939d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:54.606167 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb6bf45-ab3e-44d5-8f72-be8d4bf939d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:54.608092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb6bf45-ab3e-44d5-8f72-be8d4bf939d7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:54.644653 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2467f0f2-7203-4f0f-b70f-bf957e8586cd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:54.696310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:54.718008 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf408f-84bc-4cd1-8896-8e40b1a4b1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:54.719279 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf408f-84bc-4cd1-8896-8e40b1a4b1cb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:54.719770 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf408f-84bc-4cd1-8896-8e40b1a4b1cb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:54.720269 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf408f-84bc-4cd1-8896-8e40b1a4b1cb None admin] PATH_INFO: `/v3/projects/a19c893c70c84649bb5406d2cf9e2708` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:54.722613 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf408f-84bc-4cd1-8896-8e40b1a4b1cb None admin] RBAC: Authorizing `identity:get_project(project_id=a19c893c70c84649bb5406d2cf9e2708)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:54.724483 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf408f-84bc-4cd1-8896-8e40b1a4b1cb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:54.731467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 46/93] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:19:54 2022] GET /identity/v3/projects/a19c893c70c84649bb5406d2cf9e2708 => generated 298 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:54.737125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf408f-84bc-4cd1-8896-8e40b1a4b1cb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:54.783012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 48/94] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:19:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:19:54.784744 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6bf45-ab3e-44d5-8f72-be8d4bf939d7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:54.883429 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:54.927939 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf408f-84bc-4cd1-8896-8e40b1a4b1cb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:54.954851 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6bf45-ab3e-44d5-8f72-be8d4bf939d7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:54.986498 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08513c-49b5-4a65-be52-79d71c3c7e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:54.987182 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08513c-49b5-4a65-be52-79d71c3c7e9e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:54.987469 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08513c-49b5-4a65-be52-79d71c3c7e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:54.987749 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08513c-49b5-4a65-be52-79d71c3c7e9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:54.988596 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08513c-49b5-4a65-be52-79d71c3c7e9e None admin] RBAC: Adding query filter params (role.id=6005bd32383a4706a69108f7a2cf3ed1, scope.project.id=a19c893c70c84649bb5406d2cf9e2708, user.id=3f6a358f4bc044be94e44d1470a32c67) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:19:54.988912 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08513c-49b5-4a65-be52-79d71c3c7e9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:54.990378 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08513c-49b5-4a65-be52-79d71c3c7e9e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:54.992277 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1079e93e-8859-4b65-9c13-2af1c9fda817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:54.993165 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079e93e-8859-4b65-9c13-2af1c9fda817 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:54.993457 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079e93e-8859-4b65-9c13-2af1c9fda817 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:54.993743 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079e93e-8859-4b65-9c13-2af1c9fda817 None admin] PATH_INFO: `/v3/projects/2c3b5ee002c5407bb0c71eae9b756435` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:54.994991 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1079e93e-8859-4b65-9c13-2af1c9fda817 None admin] RBAC: Authorizing `identity:get_project(project_id=2c3b5ee002c5407bb0c71eae9b756435)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:54.996694 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1079e93e-8859-4b65-9c13-2af1c9fda817 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:54.998752 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 49/95] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:19:54 2022] GET /identity/v3/projects/2c3b5ee002c5407bb0c71eae9b756435 => generated 312 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:55.002824 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 47/96] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:19:54 2022] GET /identity/v3/role_assignments?user.id=3f6a358f4bc044be94e44d1470a32c67&scope.project.id=a19c893c70c84649bb5406d2cf9e2708&role.id=6005bd32383a4706a69108f7a2cf3ed1 => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:19:55.007633 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1079e93e-8859-4b65-9c13-2af1c9fda817 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:55.046054 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:55.147320 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1079e93e-8859-4b65-9c13-2af1c9fda817 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:55.197891 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3874d7-0ead-47b7-b047-f7dbaa30eaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:55.198596 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3874d7-0ead-47b7-b047-f7dbaa30eaef None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:55.199003 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3874d7-0ead-47b7-b047-f7dbaa30eaef None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:55.199266 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3874d7-0ead-47b7-b047-f7dbaa30eaef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:55.200123 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3874d7-0ead-47b7-b047-f7dbaa30eaef None admin] RBAC: Adding query filter params (role.id=3bf034b395b44d88b3af67da01a6ed82, scope.project.id=2c3b5ee002c5407bb0c71eae9b756435, user.id=3f6a358f4bc044be94e44d1470a32c67) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:19:55.200445 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3874d7-0ead-47b7-b047-f7dbaa30eaef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:55.201901 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3874d7-0ead-47b7-b047-f7dbaa30eaef None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:55.218243 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 50/97] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:19:55 2022] GET /identity/v3/role_assignments?user.id=3f6a358f4bc044be94e44d1470a32c67&scope.project.id=2c3b5ee002c5407bb0c71eae9b756435&role.id=3bf034b395b44d88b3af67da01a6ed82 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:56.229551 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0baaad-5c0e-428e-ab3d-4c74b44a3913 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:56.230055 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0baaad-5c0e-428e-ab3d-4c74b44a3913 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:56.230688 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0baaad-5c0e-428e-ab3d-4c74b44a3913 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:56.232270 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 48/98] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:19:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:19:56.237668 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db8446-9d62-493b-991a-10a5fcb37c4e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:56.238083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db8446-9d62-493b-991a-10a5fcb37c4e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:56.238247 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db8446-9d62-493b-991a-10a5fcb37c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:56.258660 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b8db8446-9d62-493b-991a-10a5fcb37c4e None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:19:56.259917 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b8db8446-9d62-493b-991a-10a5fcb37c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:19:56.344891 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 51/99] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:19:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:19:56.380269 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0baaad-5c0e-428e-ab3d-4c74b44a3913 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:56.511197 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0baaad-5c0e-428e-ab3d-4c74b44a3913 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:56.552357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598bd1e-7bc3-4c43-bd12-fbe657d6a4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:56.553176 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598bd1e-7bc3-4c43-bd12-fbe657d6a4a9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:56.553469 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598bd1e-7bc3-4c43-bd12-fbe657d6a4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:56.553756 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598bd1e-7bc3-4c43-bd12-fbe657d6a4a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:56.565301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:56.571501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598bd1e-7bc3-4c43-bd12-fbe657d6a4a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:19:56.581613 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-1598bd1e-7bc3-4c43-bd12-fbe657d6a4a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:19:56.581613 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:19:56.581613 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:19:56.581613 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:19:56.581613 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:19:56.581613 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:19:56.581613 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:19:56.581613 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:19:56.581613 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:19:56.581613 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:19:56.581613 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:19:56.581613 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:19:56.581613 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:19:56.581613 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:19:56.581613 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:19:56.581613 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:19:56.582743 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:19:56.582743 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:19:56.582743 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:19:56.582743 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:19:56.582743 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:19:56.582743 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:19:56.582743 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:19:56.582743 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:19:56.582743 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:19:56.582743 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:19:56.582743 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:19:56.582743 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:19:56.582743 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:19:56.582743 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:19:56.582743 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:19:56.582743 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:19:56.583747 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:19:56.583747 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:19:56.583747 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:19:56.583747 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:19:56.583747 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:19:56.583747 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:19:56.583747 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:19:56.583747 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:19:56.583747 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:19:56.583747 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:19:56.583747 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:19:56.583747 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:19:56.583747 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 49/100] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:19:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:19:56.588651 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db8446-9d62-493b-991a-10a5fcb37c4e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:56.615223 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:56.700318 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db8446-9d62-493b-991a-10a5fcb37c4e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:56.751344 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c8781-ad44-4862-bac4-edc6124ae816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:56.752038 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c8781-ad44-4862-bac4-edc6124ae816 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:56.752330 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c8781-ad44-4862-bac4-edc6124ae816 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:56.752615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c8781-ad44-4862-bac4-edc6124ae816 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:56.753382 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c8781-ad44-4862-bac4-edc6124ae816 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:19:56.753696 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c8781-ad44-4862-bac4-edc6124ae816 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:56.755296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c8781-ad44-4862-bac4-edc6124ae816 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:56.769794 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 52/101] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 12:19:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:56.775849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598bd1e-7bc3-4c43-bd12-fbe657d6a4a9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:56.912161 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598bd1e-7bc3-4c43-bd12-fbe657d6a4a9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:56.956469 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c87b4-5efb-400f-aa49-1a7ca14927b0 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:56.956799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c87b4-5efb-400f-aa49-1a7ca14927b0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:56.957043 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c87b4-5efb-400f-aa49-1a7ca14927b0 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:56.958440 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 53/102] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:19:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:19:56.963621 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa7cf8-46e1-41f4-b605-1cfb2be83291 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:56.964164 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa7cf8-46e1-41f4-b605-1cfb2be83291 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:56.964760 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa7cf8-46e1-41f4-b605-1cfb2be83291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:56.966099 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572ce73-5ce7-45de-a67d-48e3ec6cdcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:56.970443 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572ce73-5ce7-45de-a67d-48e3ec6cdcc7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:56.970443 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572ce73-5ce7-45de-a67d-48e3ec6cdcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:56.970443 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572ce73-5ce7-45de-a67d-48e3ec6cdcc7 None admin] PATH_INFO: `/v3/users/3f6a358f4bc044be94e44d1470a32c67` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:56.971166 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572ce73-5ce7-45de-a67d-48e3ec6cdcc7 None admin] RBAC: Authorizing `identity:get_user(user_id=3f6a358f4bc044be94e44d1470a32c67)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:56.972640 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572ce73-5ce7-45de-a67d-48e3ec6cdcc7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:56.974471 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 50/103] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:19:56 2022] GET /identity/v3/users/3f6a358f4bc044be94e44d1470a32c67 => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:56.979974 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572ce73-5ce7-45de-a67d-48e3ec6cdcc7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:56.999472 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c0fa7cf8-46e1-41f4-b605-1cfb2be83291 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:19:57.001249 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c0fa7cf8-46e1-41f4-b605-1cfb2be83291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:19:57.109145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:57.126634 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 54/104] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:19:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:19:57.127856 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7ff28-fe91-467a-b984-c1c805fcd844 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:57.128153 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7ff28-fe91-467a-b984-c1c805fcd844 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:57.128437 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7ff28-fe91-467a-b984-c1c805fcd844 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:57.135095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 55/105] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:19:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:19:57.136304 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95523b2-ee1b-49d1-a0e9-32db1f852f90 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:57.137137 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95523b2-ee1b-49d1-a0e9-32db1f852f90 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:57.137391 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95523b2-ee1b-49d1-a0e9-32db1f852f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:57.168267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b95523b2-ee1b-49d1-a0e9-32db1f852f90 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:19:57.169473 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b95523b2-ee1b-49d1-a0e9-32db1f852f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:19:57.182686 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572ce73-5ce7-45de-a67d-48e3ec6cdcc7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:57.182857 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:57.258876 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d5cc80-40ee-4988-b143-680a9b075602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:57.259785 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5cc80-40ee-4988-b143-680a9b075602 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:57.260138 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5cc80-40ee-4988-b143-680a9b075602 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:57.260439 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5cc80-40ee-4988-b143-680a9b075602 None admin] PATH_INFO: `/v3/projects/a19c893c70c84649bb5406d2cf9e2708` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:57.261895 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d5cc80-40ee-4988-b143-680a9b075602 None admin] RBAC: Authorizing `identity:get_project(project_id=a19c893c70c84649bb5406d2cf9e2708)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:57.263445 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d5cc80-40ee-4988-b143-680a9b075602 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:57.265279 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 51/106] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:19:56 2022] GET /identity/v3/projects/a19c893c70c84649bb5406d2cf9e2708 => generated 298 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:57.266108 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d5cc80-40ee-4988-b143-680a9b075602 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:57.302657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 56/107] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:19:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:19:57.303509 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95523b2-ee1b-49d1-a0e9-32db1f852f90 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:57.532492 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95523b2-ee1b-49d1-a0e9-32db1f852f90 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:57.541006 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d5cc80-40ee-4988-b143-680a9b075602 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:57.627679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cda623-60d0-4eef-aab5-4d57241f41f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:57.629645 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cda623-60d0-4eef-aab5-4d57241f41f3 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:57.630155 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cda623-60d0-4eef-aab5-4d57241f41f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:57.630662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cda623-60d0-4eef-aab5-4d57241f41f3 None admin] PATH_INFO: `/v3/projects/a19c893c70c84649bb5406d2cf9e2708/users/3f6a358f4bc044be94e44d1470a32c67/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:57.634622 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cda623-60d0-4eef-aab5-4d57241f41f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a19c893c70c84649bb5406d2cf9e2708, user_id=3f6a358f4bc044be94e44d1470a32c67, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:57.638076 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cda623-60d0-4eef-aab5-4d57241f41f3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:57.662306 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37745e17-f3bb-429a-8d4e-0f8c9170f3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:57.663620 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37745e17-f3bb-429a-8d4e-0f8c9170f3ee None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:57.664095 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37745e17-f3bb-429a-8d4e-0f8c9170f3ee None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:57.664605 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37745e17-f3bb-429a-8d4e-0f8c9170f3ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:57.666211 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 57/108] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 12:19:57 2022] PUT /identity/v3/projects/a19c893c70c84649bb5406d2cf9e2708/users/3f6a358f4bc044be94e44d1470a32c67/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:19:57.670814 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cda623-60d0-4eef-aab5-4d57241f41f3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:57.675537 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:57.679149 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:57.703145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37745e17-f3bb-429a-8d4e-0f8c9170f3ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:19:57.723474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-37745e17-f3bb-429a-8d4e-0f8c9170f3ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:19:57.723474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:19:57.723474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:19:57.723474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:19:57.723474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:19:57.723474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:19:57.723474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:19:57.723474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:19:57.723474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:19:57.723474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:19:57.723474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:19:57.723474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:19:57.723474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:19:57.723474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:19:57.723474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:19:57.723474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:19:57.725197 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:19:57.725197 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:19:57.725197 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:19:57.725197 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:19:57.725197 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:19:57.725197 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:19:57.725197 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:19:57.725197 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:19:57.725197 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:19:57.725197 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:19:57.725197 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:19:57.725197 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:19:57.725197 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:19:57.725197 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:19:57.725197 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:19:57.725197 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:19:57.727719 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:19:57.727719 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:19:57.727719 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:19:57.727719 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:19:57.727719 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:19:57.727719 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:19:57.727719 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:19:57.727719 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:19:57.727719 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:19:57.727719 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:19:57.727719 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:19:57.727719 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:19:57.727719 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 52/109] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:19:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 460 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:19:57.727719 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ccdb2-5eee-4cff-b2db-83fbb7d90c5b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:57.734245 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ccdb2-5eee-4cff-b2db-83fbb7d90c5b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:57.734245 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ccdb2-5eee-4cff-b2db-83fbb7d90c5b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:57.734245 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 53/110] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:19:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:19:57.738191 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ccdb2-5eee-4cff-b2db-83fbb7d90c5b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:57.887036 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cda623-60d0-4eef-aab5-4d57241f41f3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:57.909460 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ccdb2-5eee-4cff-b2db-83fbb7d90c5b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:57.939044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67442c0-9c61-4e04-a5ee-361f58815326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:57.939988 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67442c0-9c61-4e04-a5ee-361f58815326 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:57.940286 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67442c0-9c61-4e04-a5ee-361f58815326 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:57.940666 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67442c0-9c61-4e04-a5ee-361f58815326 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:57.948435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ddbdff-a76c-4dd3-9175-ce369c4c4449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:57.949630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddbdff-a76c-4dd3-9175-ce369c4c4449 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:57.949928 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddbdff-a76c-4dd3-9175-ce369c4c4449 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:57.950368 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddbdff-a76c-4dd3-9175-ce369c4c4449 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:57.951185 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ddbdff-a76c-4dd3-9175-ce369c4c4449 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:19:57.951500 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ddbdff-a76c-4dd3-9175-ce369c4c4449 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:57.952886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ddbdff-a76c-4dd3-9175-ce369c4c4449 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:57.953441 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67442c0-9c61-4e04-a5ee-361f58815326 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:19:57.961855 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 54/111] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:19:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:57.962020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e67442c0-9c61-4e04-a5ee-361f58815326 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 12:19:57.962020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:19:57.962020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:19:57.962020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:19:57.962020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:19:57.962020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:19:57.962020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:19:57.962020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:19:57.962020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:19:57.962020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:19:57.962020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:19:57.962020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:19:57.962020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:19:57.962020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:19:57.962020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:19:57.962020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:19:57.962853 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:19:57.962853 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:19:57.962853 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:19:57.962853 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:19:57.962853 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:19:57.962853 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:19:57.962853 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:19:57.962853 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:19:57.962853 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:19:57.962853 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:19:57.962853 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:19:57.962853 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:19:57.962853 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:19:57.962853 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:19:57.962853 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:19:57.962853 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:19:57.963592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:19:57.963592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:19:57.963592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:19:57.963592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:19:57.963592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:19:57.963592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:19:57.963592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:19:57.963592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:19:57.963592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:19:57.963592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:19:57.963592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 12:19:57.963592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:19:57.964177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d339ac2-0242-42ce-9fbd-d67d8d382251 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:57.964177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d339ac2-0242-42ce-9fbd-d67d8d382251 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:57.964177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d339ac2-0242-42ce-9fbd-d67d8d382251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:57.964342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 58/112] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:19:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:19:57.973372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67442c0-9c61-4e04-a5ee-361f58815326 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:57.983452 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2d339ac2-0242-42ce-9fbd-d67d8d382251 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:19:57.984768 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2d339ac2-0242-42ce-9fbd-d67d8d382251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:19:58.041324 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:58.045047 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:58.067895 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 55/113] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:19:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:19:58.068923 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d339ac2-0242-42ce-9fbd-d67d8d382251 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:58.137730 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67442c0-9c61-4e04-a5ee-361f58815326 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:58.189775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d339ac2-0242-42ce-9fbd-d67d8d382251 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:58.212225 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6aebd-a5fd-4141-8bbf-99c8f6c20f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:58.213122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6aebd-a5fd-4141-8bbf-99c8f6c20f2b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:58.213966 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6aebd-a5fd-4141-8bbf-99c8f6c20f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:58.214262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de6aebd-a5fd-4141-8bbf-99c8f6c20f2b None admin] PATH_INFO: `/v3/users/96982c6fb7694d38b8d0a9278de02353` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:58.216520 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6aebd-a5fd-4141-8bbf-99c8f6c20f2b None admin] RBAC: Authorizing `identity:get_user(user_id=96982c6fb7694d38b8d0a9278de02353)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:58.218722 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de6aebd-a5fd-4141-8bbf-99c8f6c20f2b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:58.221523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 59/114] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:19:57 2022] GET /identity/v3/users/96982c6fb7694d38b8d0a9278de02353 => generated 247 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:58.222390 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6aebd-a5fd-4141-8bbf-99c8f6c20f2b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:58.240237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2fca6-41cc-449a-9f5c-855d07fab02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:58.240917 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2fca6-41cc-449a-9f5c-855d07fab02d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:58.241220 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2fca6-41cc-449a-9f5c-855d07fab02d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:58.241491 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2fca6-41cc-449a-9f5c-855d07fab02d None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:58.242260 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2fca6-41cc-449a-9f5c-855d07fab02d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:19:58.242594 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2fca6-41cc-449a-9f5c-855d07fab02d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:58.243998 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2fca6-41cc-449a-9f5c-855d07fab02d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:58.253521 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 56/115] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 12:19:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:58.254825 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e3ee3-0e93-407d-bd77-f94085afe867 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:58.255118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e3ee3-0e93-407d-bd77-f94085afe867 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:58.255402 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76e3ee3-0e93-407d-bd77-f94085afe867 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:58.256652 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 57/116] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:19:58.260976 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e3ee3-0e93-407d-bd77-f94085afe867 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:58.333007 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:58.362655 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de6aebd-a5fd-4141-8bbf-99c8f6c20f2b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:58.399292 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:58.407450 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb075ef-183a-455f-bcf5-5dccc98e7b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:58.408754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb075ef-183a-455f-bcf5-5dccc98e7b71 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:58.409157 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb075ef-183a-455f-bcf5-5dccc98e7b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:58.409741 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb075ef-183a-455f-bcf5-5dccc98e7b71 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:58.411313 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb075ef-183a-455f-bcf5-5dccc98e7b71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:58.413138 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb075ef-183a-455f-bcf5-5dccc98e7b71 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:58.414310 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76e3ee3-0e93-407d-bd77-f94085afe867 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:58.415479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 60/117] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:19:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:58.416720 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb075ef-183a-455f-bcf5-5dccc98e7b71 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:58.459968 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36308655-1f1d-49be-883d-aef9de15e2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:58.460852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36308655-1f1d-49be-883d-aef9de15e2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:58.461144 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36308655-1f1d-49be-883d-aef9de15e2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:58.461427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36308655-1f1d-49be-883d-aef9de15e2c7 None admin] PATH_INFO: `/v3/projects/a19c893c70c84649bb5406d2cf9e2708` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:58.462638 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36308655-1f1d-49be-883d-aef9de15e2c7 None admin] RBAC: Authorizing `identity:get_project(project_id=a19c893c70c84649bb5406d2cf9e2708)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:58.464101 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36308655-1f1d-49be-883d-aef9de15e2c7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:58.465883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 58/118] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:19:58 2022] GET /identity/v3/projects/a19c893c70c84649bb5406d2cf9e2708 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:58.472018 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b17cff-07be-4756-bc90-112cb62f2159 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:58.472162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b17cff-07be-4756-bc90-112cb62f2159 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:58.472343 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b17cff-07be-4756-bc90-112cb62f2159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:58.490135 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c1b17cff-07be-4756-bc90-112cb62f2159 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:19:58.491357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c1b17cff-07be-4756-bc90-112cb62f2159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:19:58.531528 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb075ef-183a-455f-bcf5-5dccc98e7b71 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:58.565610 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 59/119] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:19:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:19:58.566250 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b17cff-07be-4756-bc90-112cb62f2159 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:58.575839 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d8ab3-9fcb-40ec-b48f-30a9188be17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:58.577578 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d8ab3-9fcb-40ec-b48f-30a9188be17e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:58.578079 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d8ab3-9fcb-40ec-b48f-30a9188be17e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:58.578875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d8ab3-9fcb-40ec-b48f-30a9188be17e None admin] PATH_INFO: `/v3/users/1399566127d74b358cd75e59fc011660` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:58.602211 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d8ab3-9fcb-40ec-b48f-30a9188be17e None admin] RBAC: Authorizing `identity:get_user(user_id=1399566127d74b358cd75e59fc011660)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:58.604023 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6d8ab3-9fcb-40ec-b48f-30a9188be17e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:58.606568 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 61/120] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:19:58 2022] GET /identity/v3/users/1399566127d74b358cd75e59fc011660 => generated 289 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:58.608816 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948143b-3fb4-40bc-b3de-7db3c7a6a083 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:58.609650 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948143b-3fb4-40bc-b3de-7db3c7a6a083 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:58.610161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948143b-3fb4-40bc-b3de-7db3c7a6a083 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:58.612384 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 62/121] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:19:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:19:58.613509 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948143b-3fb4-40bc-b3de-7db3c7a6a083 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:58.617831 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:58.672636 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b17cff-07be-4756-bc90-112cb62f2159 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:58.678398 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:58.710677 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516b61d-e360-462f-8749-3c58c096a534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:58.711357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516b61d-e360-462f-8749-3c58c096a534 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:58.711648 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516b61d-e360-462f-8749-3c58c096a534 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:58.711934 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8516b61d-e360-462f-8749-3c58c096a534 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:58.712657 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8516b61d-e360-462f-8749-3c58c096a534 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:58.715220 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8516b61d-e360-462f-8749-3c58c096a534 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:58.726696 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948143b-3fb4-40bc-b3de-7db3c7a6a083 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:58.750459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 60/122] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jun 29 12:19:58 2022] POST /identity/v3/projects => generated 302 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:19:58.750459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516b61d-e360-462f-8749-3c58c096a534 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:58.780621 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc393534-db34-4a9f-b1fe-8d1c27d80b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:58.782196 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc393534-db34-4a9f-b1fe-8d1c27d80b9a None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:58.782770 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc393534-db34-4a9f-b1fe-8d1c27d80b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:58.783058 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc393534-db34-4a9f-b1fe-8d1c27d80b9a None admin] PATH_INFO: `/v3/projects/a19c893c70c84649bb5406d2cf9e2708/users/96982c6fb7694d38b8d0a9278de02353/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:58.786325 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc393534-db34-4a9f-b1fe-8d1c27d80b9a None admin] RBAC: Authorizing `identity:create_grant(project_id=a19c893c70c84649bb5406d2cf9e2708, user_id=96982c6fb7694d38b8d0a9278de02353, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:58.788829 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc393534-db34-4a9f-b1fe-8d1c27d80b9a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:58.811491 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 63/123] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 12:19:58 2022] PUT /identity/v3/projects/a19c893c70c84649bb5406d2cf9e2708/users/96982c6fb7694d38b8d0a9278de02353/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:19:58.812750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc393534-db34-4a9f-b1fe-8d1c27d80b9a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:58.936690 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:58.945614 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc393534-db34-4a9f-b1fe-8d1c27d80b9a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:58.946486 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8516b61d-e360-462f-8749-3c58c096a534 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:58.969160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:58.989665 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070d3a0-ccdb-43c9-983f-30c5ee270abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:58.990918 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070d3a0-ccdb-43c9-983f-30c5ee270abf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:58.991496 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070d3a0-ccdb-43c9-983f-30c5ee270abf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:58.992063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070d3a0-ccdb-43c9-983f-30c5ee270abf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:58.995222 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58a24b-a029-4cc8-8ab6-a231d7b81e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:58.996579 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58a24b-a029-4cc8-8ab6-a231d7b81e7e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:58.997377 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58a24b-a029-4cc8-8ab6-a231d7b81e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:58.997670 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58a24b-a029-4cc8-8ab6-a231d7b81e7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:59.008201 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070d3a0-ccdb-43c9-983f-30c5ee270abf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:19:59.017879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d070d3a0-ccdb-43c9-983f-30c5ee270abf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 12:19:59.017879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:19:59.017879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:19:59.017879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:19:59.017879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:19:59.017879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:19:59.017879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:19:59.017879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:19:59.017879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:19:59.017879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:19:59.017879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:19:59.017879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:19:59.017879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:19:59.017879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:19:59.017879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:19:59.017879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:19:59.018858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:19:59.018858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:19:59.018858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:19:59.018858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:19:59.018858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:19:59.018858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:19:59.018858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:19:59.018858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:19:59.018858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:19:59.018858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:19:59.018858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:19:59.018858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:19:59.018858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:19:59.018858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:19:59.018858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:19:59.018858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:19:59.019623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:19:59.019623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:19:59.019623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:19:59.019623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:19:59.019623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:19:59.019623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:19:59.019623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:19:59.019623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:19:59.019623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:19:59.019623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:19:59.019623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 12:19:59.019623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:19:59.022405 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 61/124] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 12:19:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:19:59.023995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b9230-21d1-4e9e-80d9-77b4f1240cd1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:59.024481 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b9230-21d1-4e9e-80d9-77b4f1240cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:59.024951 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b9230-21d1-4e9e-80d9-77b4f1240cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:59.049597 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c88b9230-21d1-4e9e-80d9-77b4f1240cd1 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:19:59.051388 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c88b9230-21d1-4e9e-80d9-77b4f1240cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:19:59.088278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58a24b-a029-4cc8-8ab6-a231d7b81e7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:59.090547 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a58a24b-a029-4cc8-8ab6-a231d7b81e7e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:59.144380 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 62/125] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:19:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:19:59.145341 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b9230-21d1-4e9e-80d9-77b4f1240cd1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:59.222889 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 64/126] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:19:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:19:59.223752 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58a24b-a029-4cc8-8ab6-a231d7b81e7e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:59.249287 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:59.267401 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:59.284739 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b9230-21d1-4e9e-80d9-77b4f1240cd1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:59.333782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6267f-518e-4c39-8c4f-9c2b009dc9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:59.334490 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6267f-518e-4c39-8c4f-9c2b009dc9b5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:59.334783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6267f-518e-4c39-8c4f-9c2b009dc9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:59.336313 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6267f-518e-4c39-8c4f-9c2b009dc9b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:59.339202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6267f-518e-4c39-8c4f-9c2b009dc9b5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:19:59.340200 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6267f-518e-4c39-8c4f-9c2b009dc9b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:59.342204 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf6267f-518e-4c39-8c4f-9c2b009dc9b5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:59.354826 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58a24b-a029-4cc8-8ab6-a231d7b81e7e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:59.361648 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 63/127] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:19:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:59.370444 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6267f-518e-4c39-8c4f-9c2b009dc9b5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:59.415615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4a3ac-5b51-4b5a-96a7-3ab57be4a5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:59.417134 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4a3ac-5b51-4b5a-96a7-3ab57be4a5ce None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:59.418251 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4a3ac-5b51-4b5a-96a7-3ab57be4a5ce None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:59.418441 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4a3ac-5b51-4b5a-96a7-3ab57be4a5ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:59.442203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a4a3ac-5b51-4b5a-96a7-3ab57be4a5ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:19:59.451011 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e1a4a3ac-5b51-4b5a-96a7-3ab57be4a5ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:19:59.451011 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:19:59.451011 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:19:59.451011 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:19:59.451011 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:19:59.451011 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:19:59.451011 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:19:59.451011 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:19:59.451011 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:19:59.451011 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:19:59.451011 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:19:59.451011 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:19:59.451011 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:19:59.451011 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:19:59.451011 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:19:59.451011 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:19:59.451860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:19:59.451860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:19:59.451860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:19:59.451860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:19:59.451860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:19:59.451860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:19:59.451860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:19:59.451860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:19:59.451860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:19:59.451860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:19:59.451860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:19:59.451860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:19:59.451860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:19:59.451860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:19:59.451860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:19:59.451860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:19:59.452615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:19:59.452615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:19:59.452615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:19:59.452615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:19:59.452615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:19:59.452615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:19:59.452615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:19:59.452615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:19:59.452615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:19:59.452615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:19:59.452615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:19:59.452615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:19:59.452615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 65/128] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:19:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:19:59.462114 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4a3ac-5b51-4b5a-96a7-3ab57be4a5ce None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:59.553377 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6267f-518e-4c39-8c4f-9c2b009dc9b5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:59.592505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d3344-9d38-4309-b707-a72fe953b02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:59.593386 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d3344-9d38-4309-b707-a72fe953b02e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:59.593674 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d3344-9d38-4309-b707-a72fe953b02e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:59.593957 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d3344-9d38-4309-b707-a72fe953b02e None admin] PATH_INFO: `/v3/projects/a19c893c70c84649bb5406d2cf9e2708` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:59.595194 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d3344-9d38-4309-b707-a72fe953b02e None admin] RBAC: Authorizing `identity:get_project(project_id=a19c893c70c84649bb5406d2cf9e2708)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:59.596673 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d3344-9d38-4309-b707-a72fe953b02e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:59.598524 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 64/129] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:19:59 2022] GET /identity/v3/projects/a19c893c70c84649bb5406d2cf9e2708 => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:59.599381 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d3344-9d38-4309-b707-a72fe953b02e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:59.623341 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:59.625610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:19:59.665090 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4a3ac-5b51-4b5a-96a7-3ab57be4a5ce None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:59.738031 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304c20d-9eee-43c9-9ceb-a705be20b6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:59.743062 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d3344-9d38-4309-b707-a72fe953b02e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:59.743297 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304c20d-9eee-43c9-9ceb-a705be20b6dd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:59.743787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304c20d-9eee-43c9-9ceb-a705be20b6dd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:59.744257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304c20d-9eee-43c9-9ceb-a705be20b6dd None admin] PATH_INFO: `/v3/users/3f6a358f4bc044be94e44d1470a32c67` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:59.747176 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304c20d-9eee-43c9-9ceb-a705be20b6dd None admin] RBAC: Authorizing `identity:get_user(user_id=3f6a358f4bc044be94e44d1470a32c67)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:59.748918 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d304c20d-9eee-43c9-9ceb-a705be20b6dd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:59.752692 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 66/130] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:19:59 2022] GET /identity/v3/users/3f6a358f4bc044be94e44d1470a32c67 => generated 275 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:59.757361 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304c20d-9eee-43c9-9ceb-a705be20b6dd None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:59.791481 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153a8fc-b04a-49b3-a9dd-6f125d00d9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:19:59.792491 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153a8fc-b04a-49b3-a9dd-6f125d00d9cf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:19:59.793736 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153a8fc-b04a-49b3-a9dd-6f125d00d9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:19:59.794986 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153a8fc-b04a-49b3-a9dd-6f125d00d9cf None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:19:59.796070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2153a8fc-b04a-49b3-a9dd-6f125d00d9cf None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:19:59.797435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2153a8fc-b04a-49b3-a9dd-6f125d00d9cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:19:59.799330 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2153a8fc-b04a-49b3-a9dd-6f125d00d9cf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:19:59.818074 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 65/131] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 12:19:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:19:59.819875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153a8fc-b04a-49b3-a9dd-6f125d00d9cf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:19:59.959154 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d304c20d-9eee-43c9-9ceb-a705be20b6dd None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:19:59.969330 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:00.003387 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153a8fc-b04a-49b3-a9dd-6f125d00d9cf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:00.025821 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1231a3-244a-4339-98c7-3c78ef16aebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:00.026849 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1231a3-244a-4339-98c7-3c78ef16aebf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:00.027300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1231a3-244a-4339-98c7-3c78ef16aebf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:00.027596 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1231a3-244a-4339-98c7-3c78ef16aebf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:00.028455 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1231a3-244a-4339-98c7-3c78ef16aebf None admin] RBAC: Adding query filter params (role.id=c2e0da29cd7947288c717f7b4b262ef8, scope.project.id=a19c893c70c84649bb5406d2cf9e2708, user.id=1399566127d74b358cd75e59fc011660) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:00.028777 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1231a3-244a-4339-98c7-3c78ef16aebf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:00.031164 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1231a3-244a-4339-98c7-3c78ef16aebf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:00.045015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 67/132] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:19:59 2022] GET /identity/v3/role_assignments?user.id=1399566127d74b358cd75e59fc011660&scope.project.id=a19c893c70c84649bb5406d2cf9e2708&role.id=c2e0da29cd7947288c717f7b4b262ef8 => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:00.046794 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1231a3-244a-4339-98c7-3c78ef16aebf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:00.068958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c573a60-04ef-4132-8faf-39f9dc4364f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:00.069805 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c573a60-04ef-4132-8faf-39f9dc4364f1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:00.070096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c573a60-04ef-4132-8faf-39f9dc4364f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:00.070383 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c573a60-04ef-4132-8faf-39f9dc4364f1 None admin] PATH_INFO: `/v3/projects/a19c893c70c84649bb5406d2cf9e2708` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:00.071913 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:00.075247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c573a60-04ef-4132-8faf-39f9dc4364f1 None admin] RBAC: Authorizing `identity:get_project(project_id=a19c893c70c84649bb5406d2cf9e2708)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:00.076945 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c573a60-04ef-4132-8faf-39f9dc4364f1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:00.078788 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 66/133] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:19:59 2022] GET /identity/v3/projects/a19c893c70c84649bb5406d2cf9e2708 => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:00.092676 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c573a60-04ef-4132-8faf-39f9dc4364f1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:00.262990 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1231a3-244a-4339-98c7-3c78ef16aebf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:00.299505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c573a60-04ef-4132-8faf-39f9dc4364f1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:00.311336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4c7c3c-2c22-425f-b6d5-a51137d8e353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:00.311336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c7c3c-2c22-425f-b6d5-a51137d8e353 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:00.311604 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c7c3c-2c22-425f-b6d5-a51137d8e353 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:00.311829 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4c7c3c-2c22-425f-b6d5-a51137d8e353 None admin] PATH_INFO: `/v3/users/3f6a358f4bc044be94e44d1470a32c67` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:00.312992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c7c3c-2c22-425f-b6d5-a51137d8e353 None admin] RBAC: Authorizing `identity:get_user(user_id=3f6a358f4bc044be94e44d1470a32c67)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:00.314994 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4c7c3c-2c22-425f-b6d5-a51137d8e353 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:00.317902 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 68/134] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:20:00 2022] GET /identity/v3/users/3f6a358f4bc044be94e44d1470a32c67 => generated 275 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:00.320158 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd033a07-365b-4154-9150-70c3d53e8628 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:00.320654 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd033a07-365b-4154-9150-70c3d53e8628 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:00.321126 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd033a07-365b-4154-9150-70c3d53e8628 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:00.322678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 69/135] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:00.325982 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd033a07-365b-4154-9150-70c3d53e8628 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:00.376663 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463990bf-fb0b-4f12-b2c5-ad3b50b71817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:00.378063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463990bf-fb0b-4f12-b2c5-ad3b50b71817 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:00.381173 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463990bf-fb0b-4f12-b2c5-ad3b50b71817 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:00.381730 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463990bf-fb0b-4f12-b2c5-ad3b50b71817 None admin] PATH_INFO: `/v3/projects/a19c893c70c84649bb5406d2cf9e2708/users/3f6a358f4bc044be94e44d1470a32c67/roles/6005bd32383a4706a69108f7a2cf3ed1` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:00.384403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463990bf-fb0b-4f12-b2c5-ad3b50b71817 None admin] RBAC: Authorizing `identity:create_grant(project_id=a19c893c70c84649bb5406d2cf9e2708, user_id=3f6a358f4bc044be94e44d1470a32c67, role_id=6005bd32383a4706a69108f7a2cf3ed1)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:00.387898 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463990bf-fb0b-4f12-b2c5-ad3b50b71817 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:00.411691 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 67/136] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 12:20:00 2022] PUT /identity/v3/projects/a19c893c70c84649bb5406d2cf9e2708/users/3f6a358f4bc044be94e44d1470a32c67/roles/6005bd32383a4706a69108f7a2cf3ed1 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:20:00.414344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9acc6-9a9a-4232-ab28-33d4b2dcbd99 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:00.415121 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9acc6-9a9a-4232-ab28-33d4b2dcbd99 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:00.415616 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9acc6-9a9a-4232-ab28-33d4b2dcbd99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:00.450082 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8fa9acc6-9a9a-4232-ab28-33d4b2dcbd99 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:00.455667 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8fa9acc6-9a9a-4232-ab28-33d4b2dcbd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:00.462476 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:00.513191 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:00.555909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd033a07-365b-4154-9150-70c3d53e8628 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:00.601301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 68/137] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:00.616483 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a29ca4-ded5-42e1-8cf9-e5a75978e7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:00.617813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a29ca4-ded5-42e1-8cf9-e5a75978e7ea None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:00.618316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a29ca4-ded5-42e1-8cf9-e5a75978e7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:00.618871 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a29ca4-ded5-42e1-8cf9-e5a75978e7ea None admin] PATH_INFO: `/v3/projects/2c3b5ee002c5407bb0c71eae9b756435` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:00.620305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a29ca4-ded5-42e1-8cf9-e5a75978e7ea None admin] RBAC: Authorizing `identity:get_project(project_id=2c3b5ee002c5407bb0c71eae9b756435)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:00.622603 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a29ca4-ded5-42e1-8cf9-e5a75978e7ea None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:00.624588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 70/138] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:00 2022] GET /identity/v3/projects/2c3b5ee002c5407bb0c71eae9b756435 => generated 312 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:00.630678 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9acc6-9a9a-4232-ab28-33d4b2dcbd99 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:00.636793 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a29ca4-ded5-42e1-8cf9-e5a75978e7ea None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:00.800194 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9acc6-9a9a-4232-ab28-33d4b2dcbd99 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:00.806239 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a29ca4-ded5-42e1-8cf9-e5a75978e7ea None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:00.870295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb3c0d-6eb7-446d-a72a-b2724f8f294d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:00.871202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb3c0d-6eb7-446d-a72a-b2724f8f294d None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:00.871491 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb3c0d-6eb7-446d-a72a-b2724f8f294d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:00.871772 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb3c0d-6eb7-446d-a72a-b2724f8f294d None admin] PATH_INFO: `/v3/projects/2c3b5ee002c5407bb0c71eae9b756435/users/3f6a358f4bc044be94e44d1470a32c67/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:00.881097 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eb3c0d-6eb7-446d-a72a-b2724f8f294d None admin] RBAC: Authorizing `identity:create_grant(project_id=2c3b5ee002c5407bb0c71eae9b756435, user_id=3f6a358f4bc044be94e44d1470a32c67, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:00.882618 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eb3c0d-6eb7-446d-a72a-b2724f8f294d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:00.888422 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da77e3d-74c8-4855-a667-49f5d0695cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:00.889238 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da77e3d-74c8-4855-a667-49f5d0695cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:00.889532 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da77e3d-74c8-4855-a667-49f5d0695cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:00.889818 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da77e3d-74c8-4855-a667-49f5d0695cc9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:00.906839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:00.919796 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 69/139] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 12:20:00 2022] PUT /identity/v3/projects/2c3b5ee002c5407bb0c71eae9b756435/users/3f6a358f4bc044be94e44d1470a32c67/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:20:00.922172 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da77e3d-74c8-4855-a667-49f5d0695cc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:00.930365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3da77e3d-74c8-4855-a667-49f5d0695cc9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:20:00.930365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:00.930365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:00.930365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:00.930365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:00.930365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:00.930365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:00.930365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:00.930365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:00.930365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:00.930365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:00.930365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:00.930365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:00.930365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:00.930365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:00.930365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:00.931248 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:00.931248 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:00.931248 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:00.931248 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:00.931248 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:00.931248 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:00.931248 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:00.931248 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:00.931248 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:00.931248 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:00.931248 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:00.931248 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:00.931248 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:00.931248 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:00.931248 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:00.931248 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:00.932005 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:00.932005 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:00.932005 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:00.932005 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:00.932005 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:00.932005 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:00.932005 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:00.932005 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:00.932005 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:00.932005 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:00.932005 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:20:00.932005 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:00.938997 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 71/140] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:20:00.944962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb3c0d-6eb7-446d-a72a-b2724f8f294d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:01.088248 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb3c0d-6eb7-446d-a72a-b2724f8f294d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:01.133914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c597ccb-ceb2-4f0b-9a4d-1e5352d6f640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:01.134546 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c597ccb-ceb2-4f0b-9a4d-1e5352d6f640 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:01.134835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c597ccb-ceb2-4f0b-9a4d-1e5352d6f640 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:01.135118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c597ccb-ceb2-4f0b-9a4d-1e5352d6f640 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:01.135880 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c597ccb-ceb2-4f0b-9a4d-1e5352d6f640 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:01.136190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c597ccb-ceb2-4f0b-9a4d-1e5352d6f640 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:01.138247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c597ccb-ceb2-4f0b-9a4d-1e5352d6f640 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:01.148561 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 70/141] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 12:20:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:01.159612 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da77e3d-74c8-4855-a667-49f5d0695cc9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:01.164348 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:01.289040 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da77e3d-74c8-4855-a667-49f5d0695cc9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:01.332821 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deaea4d-e180-4cb4-a0e6-3ebc029293da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:01.333711 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deaea4d-e180-4cb4-a0e6-3ebc029293da None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:01.334001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deaea4d-e180-4cb4-a0e6-3ebc029293da None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:01.334284 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deaea4d-e180-4cb4-a0e6-3ebc029293da None admin] PATH_INFO: `/v3/users/3f6a358f4bc044be94e44d1470a32c67` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:01.335539 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deaea4d-e180-4cb4-a0e6-3ebc029293da None admin] RBAC: Authorizing `identity:get_user(user_id=3f6a358f4bc044be94e44d1470a32c67)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:01.337022 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deaea4d-e180-4cb4-a0e6-3ebc029293da None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:01.338779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 72/142] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:20:01 2022] GET /identity/v3/users/3f6a358f4bc044be94e44d1470a32c67 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:01.345085 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c597ccb-ceb2-4f0b-9a4d-1e5352d6f640 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:01.435235 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:01.489074 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c597ccb-ceb2-4f0b-9a4d-1e5352d6f640 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:01.543873 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357345af-5c23-4597-84d1-1bb04ce2701c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:01.544539 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357345af-5c23-4597-84d1-1bb04ce2701c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:01.544828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357345af-5c23-4597-84d1-1bb04ce2701c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:01.545110 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357345af-5c23-4597-84d1-1bb04ce2701c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:01.579817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8ceca-7dde-4829-8523-bb9159a2fa15 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:01.580338 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8ceca-7dde-4829-8523-bb9159a2fa15 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:01.580412 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8ceca-7dde-4829-8523-bb9159a2fa15 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:01.581885 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 73/143] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:20:01.592897 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290961c-0d19-404e-b582-ed1d64e6a1a5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:01.593190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290961c-0d19-404e-b582-ed1d64e6a1a5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:01.593472 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290961c-0d19-404e-b582-ed1d64e6a1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:01.623106 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c290961c-0d19-404e-b582-ed1d64e6a1a5 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:01.624622 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c290961c-0d19-404e-b582-ed1d64e6a1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:01.682536 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357345af-5c23-4597-84d1-1bb04ce2701c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:01.685460 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357345af-5c23-4597-84d1-1bb04ce2701c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:01.773016 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:01.780353 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 74/144] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:01.815072 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 71/145] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:20:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:20:01.822847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c290961c-0d19-404e-b582-ed1d64e6a1a5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:01.825826 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:01.832602 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357345af-5c23-4597-84d1-1bb04ce2701c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:01.964107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357345af-5c23-4597-84d1-1bb04ce2701c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:02.002914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda793b-be3f-4ba5-944e-6ad1d0396dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:02.003726 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda793b-be3f-4ba5-944e-6ad1d0396dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:02.004009 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda793b-be3f-4ba5-944e-6ad1d0396dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:02.004289 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda793b-be3f-4ba5-944e-6ad1d0396dc4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:02.015507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda793b-be3f-4ba5-944e-6ad1d0396dc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:02.024290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-8dda793b-be3f-4ba5-944e-6ad1d0396dc4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:20:02.024290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:02.024290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:02.024290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:02.024290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:02.024290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:02.024290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:02.024290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:02.024290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:02.024290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:02.024290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:02.024290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:02.024290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:02.024290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:02.024290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:02.024290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:02.025145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:02.025145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:02.025145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:02.025145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:02.025145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:02.025145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:02.025145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:02.025145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:02.025145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:02.025145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:02.025145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:02.025145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:02.025145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:02.025145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:02.025145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:02.025145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:02.025896 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:02.025896 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:02.025896 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:02.025896 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:02.025896 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:02.025896 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:02.025896 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:02.025896 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:02.025896 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:02.025896 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:02.025896 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:20:02.025896 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:02.025896 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 72/146] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:20:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:20:02.030452 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1aa83-3d81-48fd-932d-130734a70763 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:02.030452 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1aa83-3d81-48fd-932d-130734a70763 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:02.030452 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1aa83-3d81-48fd-932d-130734a70763 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:02.030452 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 73/147] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:02.033636 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1aa83-3d81-48fd-932d-130734a70763 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:02.062827 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:02.118905 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c290961c-0d19-404e-b582-ed1d64e6a1a5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:02.171177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f29313-5199-4262-bba1-946ae4eb1f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:02.172061 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f29313-5199-4262-bba1-946ae4eb1f3a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:02.172354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f29313-5199-4262-bba1-946ae4eb1f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:02.172638 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f29313-5199-4262-bba1-946ae4eb1f3a None admin] PATH_INFO: `/v3/projects/a19c893c70c84649bb5406d2cf9e2708` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:02.173864 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f29313-5199-4262-bba1-946ae4eb1f3a None admin] RBAC: Authorizing `identity:get_project(project_id=a19c893c70c84649bb5406d2cf9e2708)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:02.174723 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1aa83-3d81-48fd-932d-130734a70763 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:02.175386 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f29313-5199-4262-bba1-946ae4eb1f3a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:02.177128 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 75/148] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:01 2022] GET /identity/v3/projects/a19c893c70c84649bb5406d2cf9e2708 => generated 298 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:02.178336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aafb68-5eff-4f42-946c-c77b37d7bd35 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:02.178655 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aafb68-5eff-4f42-946c-c77b37d7bd35 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:02.178941 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aafb68-5eff-4f42-946c-c77b37d7bd35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:02.215814 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50a510-cf2c-46cc-bc1a-7d26673e2a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:02.216470 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50a510-cf2c-46cc-bc1a-7d26673e2a05 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:02.216757 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50a510-cf2c-46cc-bc1a-7d26673e2a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:02.217086 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c50a510-cf2c-46cc-bc1a-7d26673e2a05 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:02.217897 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50a510-cf2c-46cc-bc1a-7d26673e2a05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:02.218214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50a510-cf2c-46cc-bc1a-7d26673e2a05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:02.221581 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c50a510-cf2c-46cc-bc1a-7d26673e2a05 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:02.226305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-17aafb68-5eff-4f42-946c-c77b37d7bd35 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:02.227538 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-17aafb68-5eff-4f42-946c-c77b37d7bd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:02.238796 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 74/149] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:02.243459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50a510-cf2c-46cc-bc1a-7d26673e2a05 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:02.356164 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:02.365761 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 76/150] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:02.366657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aafb68-5eff-4f42-946c-c77b37d7bd35 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:02.421173 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c50a510-cf2c-46cc-bc1a-7d26673e2a05 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:02.434848 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:02.457806 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c5faa-f611-461c-af4c-4d2ae38af249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:02.458622 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c5faa-f611-461c-af4c-4d2ae38af249 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:02.458910 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c5faa-f611-461c-af4c-4d2ae38af249 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:02.459236 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c5faa-f611-461c-af4c-4d2ae38af249 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:02.460071 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c5faa-f611-461c-af4c-4d2ae38af249 None admin] RBAC: Adding query filter params (role.id=3bf034b395b44d88b3af67da01a6ed82, scope.project.id=a19c893c70c84649bb5406d2cf9e2708, user.id=3f6a358f4bc044be94e44d1470a32c67) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:02.460437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c5faa-f611-461c-af4c-4d2ae38af249 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:02.461970 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c5faa-f611-461c-af4c-4d2ae38af249 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:02.473193 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 75/151] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:20:02 2022] GET /identity/v3/role_assignments?user.id=3f6a358f4bc044be94e44d1470a32c67&scope.project.id=a19c893c70c84649bb5406d2cf9e2708&role.id=3bf034b395b44d88b3af67da01a6ed82 => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:02.474058 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c5faa-f611-461c-af4c-4d2ae38af249 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:02.583834 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c5faa-f611-461c-af4c-4d2ae38af249 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:02.635457 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0272769-bf73-49eb-95f1-1dade5fa1fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:02.636968 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0272769-bf73-49eb-95f1-1dade5fa1fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:02.637264 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0272769-bf73-49eb-95f1-1dade5fa1fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:02.637660 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0272769-bf73-49eb-95f1-1dade5fa1fb8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:02.639320 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0272769-bf73-49eb-95f1-1dade5fa1fb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:02.640842 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0272769-bf73-49eb-95f1-1dade5fa1fb8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:02.651060 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 76/152] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:02.656770 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0272769-bf73-49eb-95f1-1dade5fa1fb8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:02.685284 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aafb68-5eff-4f42-946c-c77b37d7bd35 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:02.690818 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:02.752815 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e49e08-45ce-4273-8656-f31b1412383a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:02.755051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e49e08-45ce-4273-8656-f31b1412383a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:02.758615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e49e08-45ce-4273-8656-f31b1412383a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:02.759155 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e49e08-45ce-4273-8656-f31b1412383a None admin] PATH_INFO: `/v3/users/96982c6fb7694d38b8d0a9278de02353` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:02.760785 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e49e08-45ce-4273-8656-f31b1412383a None admin] RBAC: Authorizing `identity:get_user(user_id=96982c6fb7694d38b8d0a9278de02353)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:02.763894 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e49e08-45ce-4273-8656-f31b1412383a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:02.767365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 77/153] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:20:02 2022] GET /identity/v3/users/96982c6fb7694d38b8d0a9278de02353 => generated 247 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:02.779518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e49e08-45ce-4273-8656-f31b1412383a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:02.855335 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0272769-bf73-49eb-95f1-1dade5fa1fb8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:02.892629 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:02.912960 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a1060-c620-4fb5-8b91-0d16b2f58aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:02.913636 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a1060-c620-4fb5-8b91-0d16b2f58aca None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:02.914024 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a1060-c620-4fb5-8b91-0d16b2f58aca None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:02.914312 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a1060-c620-4fb5-8b91-0d16b2f58aca None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:02.915070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a1060-c620-4fb5-8b91-0d16b2f58aca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:02.916631 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a1060-c620-4fb5-8b91-0d16b2f58aca None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:02.946166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e49e08-45ce-4273-8656-f31b1412383a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:02.959643 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 77/154] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 12:20:02 2022] POST /identity/v3/users => generated 283 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:20:02.967436 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81f25d-7b68-42c3-88bf-1e0db1cd5928 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:02.967803 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81f25d-7b68-42c3-88bf-1e0db1cd5928 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:02.968089 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81f25d-7b68-42c3-88bf-1e0db1cd5928 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:02.969683 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 78/155] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:02 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:02.979025 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b362c-f91e-4fe8-81f5-94c92cbf9de1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:02.979543 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b362c-f91e-4fe8-81f5-94c92cbf9de1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:02.979831 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b362c-f91e-4fe8-81f5-94c92cbf9de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:03.013219 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e94b362c-f91e-4fe8-81f5-94c92cbf9de1 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:03.014362 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e94b362c-f91e-4fe8-81f5-94c92cbf9de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:03.029875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f16cf8d-997c-422d-801a-076f7c227538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:03.030642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16cf8d-997c-422d-801a-076f7c227538 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:03.030943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16cf8d-997c-422d-801a-076f7c227538 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:03.031228 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16cf8d-997c-422d-801a-076f7c227538 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:03.141380 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 79/156] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:20:03.165252 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b362c-f91e-4fe8-81f5-94c92cbf9de1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:03.168502 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:03.187602 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f16cf8d-997c-422d-801a-076f7c227538 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:03.189435 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f16cf8d-997c-422d-801a-076f7c227538 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:03.264411 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:03.289881 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b362c-f91e-4fe8-81f5-94c92cbf9de1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:03.305431 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 78/157] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:20:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:20:03.305431 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e661e-ef92-4bfa-989d-ea96d9d82a75 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:03.305794 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e661e-ef92-4bfa-989d-ea96d9d82a75 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:03.305916 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491e661e-ef92-4bfa-989d-ea96d9d82a75 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:03.307544 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 79/158] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:03.313558 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb0b5b-63f1-4023-8495-284a0b22ccca None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:03.313852 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb0b5b-63f1-4023-8495-284a0b22ccca None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:03.314136 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb0b5b-63f1-4023-8495-284a0b22ccca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:03.334397 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ddcb0b5b-63f1-4023-8495-284a0b22ccca None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:03.335797 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ddcb0b5b-63f1-4023-8495-284a0b22ccca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:03.337190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba07e6-db9a-40b1-9ed9-42a4a70d09c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:03.338161 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba07e6-db9a-40b1-9ed9-42a4a70d09c1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:03.338465 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba07e6-db9a-40b1-9ed9-42a4a70d09c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:03.340234 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba07e6-db9a-40b1-9ed9-42a4a70d09c1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:03.361119 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba07e6-db9a-40b1-9ed9-42a4a70d09c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:03.369549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-98ba07e6-db9a-40b1-9ed9-42a4a70d09c1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 12:20:03.369549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:03.369549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:03.369549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:03.369549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:03.369549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:03.369549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:03.369549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:03.369549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:03.369549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:03.369549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:03.369549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:03.369549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:03.369549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:03.369549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:03.369549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:03.371398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:03.371398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:03.371398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:03.371398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:03.371398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:03.371398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:03.371398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:03.371398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:03.371398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:03.371398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:03.371398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:03.371398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:03.371398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:03.371398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:03.371398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:03.371398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:03.372428 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:03.372428 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:03.372428 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:03.372428 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:03.372428 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:03.372428 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:03.372428 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:03.372428 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:03.372428 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:03.372428 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:03.372428 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 12:20:03.372428 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:03.372428 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 80/159] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:03.373498 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba07e6-db9a-40b1-9ed9-42a4a70d09c1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:03.443202 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 80/160] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:20:03.444066 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb0b5b-63f1-4023-8495-284a0b22ccca None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:03.445997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:03.504174 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba07e6-db9a-40b1-9ed9-42a4a70d09c1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:03.555152 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e732bbfc-3322-4e5b-acaa-506a03904b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:03.556943 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732bbfc-3322-4e5b-acaa-506a03904b31 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:03.557371 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732bbfc-3322-4e5b-acaa-506a03904b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:03.558047 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e732bbfc-3322-4e5b-acaa-506a03904b31 None admin] PATH_INFO: `/v3/projects/a19c893c70c84649bb5406d2cf9e2708` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:03.560712 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e732bbfc-3322-4e5b-acaa-506a03904b31 None admin] RBAC: Authorizing `identity:get_project(project_id=a19c893c70c84649bb5406d2cf9e2708)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:03.562145 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:03.563647 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e732bbfc-3322-4e5b-acaa-506a03904b31 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:03.566071 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 81/161] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:03 2022] GET /identity/v3/projects/a19c893c70c84649bb5406d2cf9e2708 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:03.567389 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e732bbfc-3322-4e5b-acaa-506a03904b31 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:03.572822 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb0b5b-63f1-4023-8495-284a0b22ccca None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:03.611451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28139fb5-b4ef-4383-b301-01199a45dbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:03.612198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28139fb5-b4ef-4383-b301-01199a45dbd8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:03.612436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28139fb5-b4ef-4383-b301-01199a45dbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:03.612716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28139fb5-b4ef-4383-b301-01199a45dbd8 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:03.613468 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28139fb5-b4ef-4383-b301-01199a45dbd8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:03.613969 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28139fb5-b4ef-4383-b301-01199a45dbd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:03.615200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28139fb5-b4ef-4383-b301-01199a45dbd8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:03.624972 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 81/162] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 12:20:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:03.626025 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28139fb5-b4ef-4383-b301-01199a45dbd8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:03.713307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e732bbfc-3322-4e5b-acaa-506a03904b31 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:03.740968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28139fb5-b4ef-4383-b301-01199a45dbd8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:03.748721 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:03.755449 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61de1f-d9c2-44c5-8979-69765920228a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:03.756328 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61de1f-d9c2-44c5-8979-69765920228a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:03.756553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61de1f-d9c2-44c5-8979-69765920228a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:03.756838 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61de1f-d9c2-44c5-8979-69765920228a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:03.769959 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb61de1f-d9c2-44c5-8979-69765920228a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:03.780746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-eb61de1f-d9c2-44c5-8979-69765920228a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 12:20:03.780746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:03.780746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:03.780746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:03.780746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:03.780746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:03.780746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:03.780746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:03.780746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:03.780746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:03.780746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:03.780746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:03.780746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:03.780746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:03.780746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:03.780746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:03.781878 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:03.781878 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:03.781878 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:03.781878 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:03.781878 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:03.781878 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:03.781878 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:03.781878 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:03.781878 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:03.781878 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:03.781878 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:03.781878 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:03.781878 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:03.781878 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:03.781878 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:03.781878 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:03.788107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:03.788107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:03.788107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:03.788107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:03.788107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:03.788107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:03.788107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:03.788107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:03.788107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:03.788107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:03.788107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 12:20:03.788107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:03.788107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 82/163] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 12:20:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:03.788107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61de1f-d9c2-44c5-8979-69765920228a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:03.788992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f33dd-b436-4969-b043-0507cb52fbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:03.788992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f33dd-b436-4969-b043-0507cb52fbcf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:03.788992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f33dd-b436-4969-b043-0507cb52fbcf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:03.788992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f33dd-b436-4969-b043-0507cb52fbcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:03.789285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f33dd-b436-4969-b043-0507cb52fbcf None admin] RBAC: Adding query filter params (role.id=3a7f4bf556344407a6d9540c89d13641, scope.project.id=a19c893c70c84649bb5406d2cf9e2708, user.id=96982c6fb7694d38b8d0a9278de02353) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:03.789434 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f33dd-b436-4969-b043-0507cb52fbcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:03.790932 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f33dd-b436-4969-b043-0507cb52fbcf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:03.799041 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 82/164] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:20:03 2022] GET /identity/v3/role_assignments?user.id=96982c6fb7694d38b8d0a9278de02353&scope.project.id=a19c893c70c84649bb5406d2cf9e2708&role.id=3a7f4bf556344407a6d9540c89d13641 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:03.801747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f33dd-b436-4969-b043-0507cb52fbcf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:03.832889 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:03.915373 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61de1f-d9c2-44c5-8979-69765920228a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:03.918657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f33dd-b436-4969-b043-0507cb52fbcf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:03.955309 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b861f-14ec-4b98-baaf-605d0ef08260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:03.956564 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b861f-14ec-4b98-baaf-605d0ef08260 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:03.957003 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b861f-14ec-4b98-baaf-605d0ef08260 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:03.957521 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b861f-14ec-4b98-baaf-605d0ef08260 None admin] PATH_INFO: `/v3/users/1399566127d74b358cd75e59fc011660` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:03.958990 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b861f-14ec-4b98-baaf-605d0ef08260 None admin] RBAC: Authorizing `identity:get_user(user_id=1399566127d74b358cd75e59fc011660)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:03.960725 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b861f-14ec-4b98-baaf-605d0ef08260 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:03.963012 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 83/165] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:20:03 2022] GET /identity/v3/users/1399566127d74b358cd75e59fc011660 => generated 289 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:03.968339 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de7d95-0dcd-4a57-a056-d4acc4c9a4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:03.969039 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de7d95-0dcd-4a57-a056-d4acc4c9a4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:03.969153 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b861f-14ec-4b98-baaf-605d0ef08260 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:03.969326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de7d95-0dcd-4a57-a056-d4acc4c9a4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:03.969606 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de7d95-0dcd-4a57-a056-d4acc4c9a4d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:03.970501 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de7d95-0dcd-4a57-a056-d4acc4c9a4d1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:03.970817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de7d95-0dcd-4a57-a056-d4acc4c9a4d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:03.972219 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de7d95-0dcd-4a57-a056-d4acc4c9a4d1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:03.985781 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 83/166] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:20:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:03.991307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de7d95-0dcd-4a57-a056-d4acc4c9a4d1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:04.047142 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:04.085945 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b861f-14ec-4b98-baaf-605d0ef08260 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:04.117491 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:04.135314 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de7d95-0dcd-4a57-a056-d4acc4c9a4d1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:04.142230 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793806c-6156-4d11-a84d-ec93758ecc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:04.143425 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793806c-6156-4d11-a84d-ec93758ecc7b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:04.144006 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793806c-6156-4d11-a84d-ec93758ecc7b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:04.144445 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793806c-6156-4d11-a84d-ec93758ecc7b None admin] PATH_INFO: `/v3/projects/a19c893c70c84649bb5406d2cf9e2708` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:04.147623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4793806c-6156-4d11-a84d-ec93758ecc7b None admin] RBAC: Authorizing `identity:get_project(project_id=a19c893c70c84649bb5406d2cf9e2708)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:04.149343 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4793806c-6156-4d11-a84d-ec93758ecc7b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:04.151571 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 84/167] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:03 2022] GET /identity/v3/projects/a19c893c70c84649bb5406d2cf9e2708 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:04.162447 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af34544-551a-41ae-ab3a-c61d943fc11a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:04.163238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af34544-551a-41ae-ab3a-c61d943fc11a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:04.163679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af34544-551a-41ae-ab3a-c61d943fc11a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:04.165762 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 85/168] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:04 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:04.166931 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af34544-551a-41ae-ab3a-c61d943fc11a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:04.174292 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55e7b3-70f3-4532-97f9-379af4358e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:04.175232 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55e7b3-70f3-4532-97f9-379af4358e7d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:04.175518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55e7b3-70f3-4532-97f9-379af4358e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:04.175799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55e7b3-70f3-4532-97f9-379af4358e7d None admin] PATH_INFO: `/v3/users/3f6a358f4bc044be94e44d1470a32c67` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:04.177029 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec55e7b3-70f3-4532-97f9-379af4358e7d None admin] RBAC: Authorizing `identity:get_user(user_id=3f6a358f4bc044be94e44d1470a32c67)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:04.182476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec55e7b3-70f3-4532-97f9-379af4358e7d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:04.185358 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 84/169] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:20:03 2022] GET /identity/v3/users/3f6a358f4bc044be94e44d1470a32c67 => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:04.187261 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9fa41-a3cd-4c46-ba2c-72a770c8ca04 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:04.187261 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9fa41-a3cd-4c46-ba2c-72a770c8ca04 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:04.187475 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9fa41-a3cd-4c46-ba2c-72a770c8ca04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:04.210498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-edc9fa41-a3cd-4c46-ba2c-72a770c8ca04 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:04.210498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-edc9fa41-a3cd-4c46-ba2c-72a770c8ca04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:04.282453 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 85/170] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:04.282453 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9fa41-a3cd-4c46-ba2c-72a770c8ca04 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:04.353436 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af34544-551a-41ae-ab3a-c61d943fc11a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:04.378542 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:04.390826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:04.403810 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d3465-677c-49c2-a5b8-09dcbfe8df08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:04.405077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d3465-677c-49c2-a5b8-09dcbfe8df08 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:04.405534 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d3465-677c-49c2-a5b8-09dcbfe8df08 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:04.405960 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17d3465-677c-49c2-a5b8-09dcbfe8df08 None admin] PATH_INFO: `/v3/projects/a19c893c70c84649bb5406d2cf9e2708/users/1399566127d74b358cd75e59fc011660/roles/c2e0da29cd7947288c717f7b4b262ef8` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:04.408372 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d3465-677c-49c2-a5b8-09dcbfe8df08 None admin] RBAC: Authorizing `identity:create_grant(project_id=a19c893c70c84649bb5406d2cf9e2708, user_id=1399566127d74b358cd75e59fc011660, role_id=c2e0da29cd7947288c717f7b4b262ef8)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:04.410095 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17d3465-677c-49c2-a5b8-09dcbfe8df08 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:04.413844 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9fa41-a3cd-4c46-ba2c-72a770c8ca04 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:04.425734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 86/171] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 12:20:04 2022] PUT /identity/v3/projects/a19c893c70c84649bb5406d2cf9e2708/users/1399566127d74b358cd75e59fc011660/roles/c2e0da29cd7947288c717f7b4b262ef8 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:20:04.426909 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d3465-677c-49c2-a5b8-09dcbfe8df08 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:04.472176 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f7c7f-d4ef-4a65-b7bd-a17528da5d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:04.472857 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f7c7f-d4ef-4a65-b7bd-a17528da5d18 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:04.473148 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f7c7f-d4ef-4a65-b7bd-a17528da5d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:04.478508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f7c7f-d4ef-4a65-b7bd-a17528da5d18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:04.604358 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f7c7f-d4ef-4a65-b7bd-a17528da5d18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:04.606681 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443f7c7f-d4ef-4a65-b7bd-a17528da5d18 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:04.609339 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17d3465-677c-49c2-a5b8-09dcbfe8df08 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:04.694807 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ddbc3-31e0-4078-8f11-bd2b818b83a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:04.695616 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ddbc3-31e0-4078-8f11-bd2b818b83a3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:04.695904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ddbc3-31e0-4078-8f11-bd2b818b83a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:04.696185 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ddbc3-31e0-4078-8f11-bd2b818b83a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:04.713562 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:04.723613 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ddbc3-31e0-4078-8f11-bd2b818b83a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:04.735978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-147ddbc3-31e0-4078-8f11-bd2b818b83a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:20:04.735978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:04.735978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:04.735978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:04.735978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:04.735978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:04.735978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:04.735978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:04.735978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:04.735978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:04.735978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:04.735978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:04.735978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:04.735978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:04.735978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:04.735978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:04.736997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:04.736997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:04.736997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:04.736997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:04.736997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:04.736997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:04.736997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:04.736997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:04.736997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:04.736997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:04.736997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:04.736997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:04.736997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:04.736997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:04.736997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:04.736997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:04.737797 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:04.737797 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:04.737797 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:04.737797 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:04.737797 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:04.737797 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:04.737797 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:04.737797 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:04.737797 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:04.737797 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:04.737797 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:20:04.737797 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:04.737797 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 87/172] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:04.739590 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 86/173] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:20:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:20:04.741456 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ddbc3-31e0-4078-8f11-bd2b818b83a3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:04.744103 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f7c7f-d4ef-4a65-b7bd-a17528da5d18 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:04.798845 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:04.890533 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f7c7f-d4ef-4a65-b7bd-a17528da5d18 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:04.896500 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ddbc3-31e0-4078-8f11-bd2b818b83a3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:04.926445 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddbe9d2-5696-4076-adfc-999f7cb95d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:04.926445 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbe9d2-5696-4076-adfc-999f7cb95d7f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:04.926445 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbe9d2-5696-4076-adfc-999f7cb95d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:04.926445 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbe9d2-5696-4076-adfc-999f7cb95d7f None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:04.926848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddbe9d2-5696-4076-adfc-999f7cb95d7f None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:04.926848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddbe9d2-5696-4076-adfc-999f7cb95d7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:04.926848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddbe9d2-5696-4076-adfc-999f7cb95d7f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:04.936126 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 87/174] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 12:20:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:04.941741 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddbe9d2-5696-4076-adfc-999f7cb95d7f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:04.952548 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaef674-9a69-4e15-aa7f-de4bd19f1863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:04.953782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaef674-9a69-4e15-aa7f-de4bd19f1863 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:04.954275 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaef674-9a69-4e15-aa7f-de4bd19f1863 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:04.954789 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaef674-9a69-4e15-aa7f-de4bd19f1863 None admin] PATH_INFO: `/v3/projects/a19c893c70c84649bb5406d2cf9e2708` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:04.956261 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaef674-9a69-4e15-aa7f-de4bd19f1863 None admin] RBAC: Authorizing `identity:get_project(project_id=a19c893c70c84649bb5406d2cf9e2708)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:04.957997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaef674-9a69-4e15-aa7f-de4bd19f1863 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:04.959953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 88/175] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:04 2022] GET /identity/v3/projects/a19c893c70c84649bb5406d2cf9e2708 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:04.966652 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaef674-9a69-4e15-aa7f-de4bd19f1863 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:05.026607 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:05.114691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddbe9d2-5696-4076-adfc-999f7cb95d7f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:05.157287 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d2d64-8789-4c35-befc-28ffdd6781c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:05.158096 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d2d64-8789-4c35-befc-28ffdd6781c2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:05.158376 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d2d64-8789-4c35-befc-28ffdd6781c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:05.158673 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d2d64-8789-4c35-befc-28ffdd6781c2 None admin] PATH_INFO: `/v3/users/3f6a358f4bc044be94e44d1470a32c67` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:05.161579 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d2d64-8789-4c35-befc-28ffdd6781c2 None admin] RBAC: Authorizing `identity:get_user(user_id=3f6a358f4bc044be94e44d1470a32c67)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:05.163113 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d2d64-8789-4c35-befc-28ffdd6781c2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:05.164749 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 88/176] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:20:04 2022] GET /identity/v3/users/3f6a358f4bc044be94e44d1470a32c67 => generated 275 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:05.171790 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d2d64-8789-4c35-befc-28ffdd6781c2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:05.214523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaef674-9a69-4e15-aa7f-de4bd19f1863 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:05.246773 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:05.283512 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b670dd4-662c-495c-8e77-eafe94c44bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:05.284174 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b670dd4-662c-495c-8e77-eafe94c44bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:05.284456 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b670dd4-662c-495c-8e77-eafe94c44bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:05.284736 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b670dd4-662c-495c-8e77-eafe94c44bd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:05.285577 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b670dd4-662c-495c-8e77-eafe94c44bd0 None admin] RBAC: Adding query filter params (role.id=6005bd32383a4706a69108f7a2cf3ed1, scope.project.id=a19c893c70c84649bb5406d2cf9e2708, user.id=3f6a358f4bc044be94e44d1470a32c67) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:05.285890 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b670dd4-662c-495c-8e77-eafe94c44bd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:05.287361 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b670dd4-662c-495c-8e77-eafe94c44bd0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:05.305922 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 89/177] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:20:04 2022] GET /identity/v3/role_assignments?user.id=3f6a358f4bc044be94e44d1470a32c67&scope.project.id=a19c893c70c84649bb5406d2cf9e2708&role.id=6005bd32383a4706a69108f7a2cf3ed1 => generated 608 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:05.331952 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d2d64-8789-4c35-befc-28ffdd6781c2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:05.347132 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:05.370383 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af7e18f-ad0e-41c0-9296-ec2942d928c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:05.371146 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7e18f-ad0e-41c0-9296-ec2942d928c5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:05.371431 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7e18f-ad0e-41c0-9296-ec2942d928c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:05.371865 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af7e18f-ad0e-41c0-9296-ec2942d928c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:05.446769 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af7e18f-ad0e-41c0-9296-ec2942d928c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:05.448337 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af7e18f-ad0e-41c0-9296-ec2942d928c5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:05.560906 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 89/178] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:20:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:20:05.566579 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b670dd4-662c-495c-8e77-eafe94c44bd0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:05.692572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b670dd4-662c-495c-8e77-eafe94c44bd0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:05.732642 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad4809-2d2c-4e9a-9449-f11b5cae05ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:05.733436 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad4809-2d2c-4e9a-9449-f11b5cae05ca None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:05.733718 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad4809-2d2c-4e9a-9449-f11b5cae05ca None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:05.733999 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad4809-2d2c-4e9a-9449-f11b5cae05ca None admin] PATH_INFO: `/v3/projects/2c3b5ee002c5407bb0c71eae9b756435` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:05.735223 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad4809-2d2c-4e9a-9449-f11b5cae05ca None admin] RBAC: Authorizing `identity:get_project(project_id=2c3b5ee002c5407bb0c71eae9b756435)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:05.736674 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad4809-2d2c-4e9a-9449-f11b5cae05ca None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:05.738377 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 90/179] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:05 2022] GET /identity/v3/projects/2c3b5ee002c5407bb0c71eae9b756435 => generated 312 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:05.744274 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af7e18f-ad0e-41c0-9296-ec2942d928c5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:05.760577 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:05.852049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af7e18f-ad0e-41c0-9296-ec2942d928c5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:05.887335 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43cae72-6826-41f8-9805-063c02b75664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:05.887993 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43cae72-6826-41f8-9805-063c02b75664 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:05.888276 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43cae72-6826-41f8-9805-063c02b75664 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:05.888553 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43cae72-6826-41f8-9805-063c02b75664 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:05.889388 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43cae72-6826-41f8-9805-063c02b75664 None admin] RBAC: Adding query filter params (role.id=3bf034b395b44d88b3af67da01a6ed82, scope.project.id=2c3b5ee002c5407bb0c71eae9b756435, user.id=3f6a358f4bc044be94e44d1470a32c67) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:05.889699 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43cae72-6826-41f8-9805-063c02b75664 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:05.891156 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43cae72-6826-41f8-9805-063c02b75664 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:05.897582 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 90/180] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:20:05 2022] GET /identity/v3/role_assignments?user.id=3f6a358f4bc044be94e44d1470a32c67&scope.project.id=2c3b5ee002c5407bb0c71eae9b756435&role.id=3bf034b395b44d88b3af67da01a6ed82 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:06.158899 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7306f7-30f8-4dca-ba9d-b9a50eff26a3 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:06.159403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7306f7-30f8-4dca-ba9d-b9a50eff26a3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:06.159508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7306f7-30f8-4dca-ba9d-b9a50eff26a3 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:06.160899 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 91/181] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:06.168973 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b953478-a675-4f80-b36b-e3a2c36dd153 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:06.169260 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b953478-a675-4f80-b36b-e3a2c36dd153 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:06.169540 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b953478-a675-4f80-b36b-e3a2c36dd153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:06.197414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4b953478-a675-4f80-b36b-e3a2c36dd153 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:06.199475 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4b953478-a675-4f80-b36b-e3a2c36dd153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:06.276937 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 91/182] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:20:06.300330 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7306f7-30f8-4dca-ba9d-b9a50eff26a3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:06.352514 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:06.414881 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7306f7-30f8-4dca-ba9d-b9a50eff26a3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:06.449458 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78dd511-54b4-461c-af60-c577a9b33133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:06.450266 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78dd511-54b4-461c-af60-c577a9b33133 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:06.450582 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78dd511-54b4-461c-af60-c577a9b33133 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:06.451015 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78dd511-54b4-461c-af60-c577a9b33133 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:06.461758 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78dd511-54b4-461c-af60-c577a9b33133 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:06.470159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b78dd511-54b4-461c-af60-c577a9b33133 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:20:06.470159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:06.470159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:06.470159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:06.470159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:06.470159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:06.470159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:06.470159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:06.470159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:06.470159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:06.470159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:06.470159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:06.470159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:06.470159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:06.470159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:06.470159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:06.472254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:06.472254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:06.472254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:06.472254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:06.472254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:06.472254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:06.472254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:06.472254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:06.472254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:06.472254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:06.472254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:06.472254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:06.472254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:06.472254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:06.472254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:06.472254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:06.473611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:06.473611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:06.473611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:06.473611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:06.473611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:06.473611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:06.473611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:06.473611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:06.473611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:06.473611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:06.473611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:20:06.473611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:06.473611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 92/183] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:20:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:06.477377 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b953478-a675-4f80-b36b-e3a2c36dd153 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:06.488404 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:06.583725 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b953478-a675-4f80-b36b-e3a2c36dd153 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:06.623976 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98c7f59-9d14-4ca5-b353-52ff20459225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:06.624665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c7f59-9d14-4ca5-b353-52ff20459225 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:06.624954 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c7f59-9d14-4ca5-b353-52ff20459225 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:06.625233 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98c7f59-9d14-4ca5-b353-52ff20459225 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:06.625992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98c7f59-9d14-4ca5-b353-52ff20459225 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:06.626309 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98c7f59-9d14-4ca5-b353-52ff20459225 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:06.627719 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98c7f59-9d14-4ca5-b353-52ff20459225 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:06.636294 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 92/184] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:06.642009 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78dd511-54b4-461c-af60-c577a9b33133 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:06.671220 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35ce23-ebd0-428a-b01d-d703144b7c74 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:06.671504 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35ce23-ebd0-428a-b01d-d703144b7c74 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:06.671857 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35ce23-ebd0-428a-b01d-d703144b7c74 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:06.673956 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 93/185] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:06.680604 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf851bba-8608-42bd-b539-e6622529554f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:06.680895 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf851bba-8608-42bd-b539-e6622529554f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:06.681181 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf851bba-8608-42bd-b539-e6622529554f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:06.703797 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bf851bba-8608-42bd-b539-e6622529554f None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:06.704994 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bf851bba-8608-42bd-b539-e6622529554f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:06.755734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78dd511-54b4-461c-af60-c577a9b33133 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:06.763116 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:06.791365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:06.797479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 94/186] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:20:06.797956 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d58f26-035e-4ef9-adb4-c911bea558a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:06.798628 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44beb4cd-cfee-4924-96d8-150d4aeb7b7d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:06.798925 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44beb4cd-cfee-4924-96d8-150d4aeb7b7d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:06.799215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44beb4cd-cfee-4924-96d8-150d4aeb7b7d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:06.799553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d58f26-035e-4ef9-adb4-c911bea558a1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:06.799965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d58f26-035e-4ef9-adb4-c911bea558a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:06.800365 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d58f26-035e-4ef9-adb4-c911bea558a1 None admin] PATH_INFO: `/v3/users/94416083e41b46c798a997615dd9b5ec` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:06.800662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 95/187] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:06.805768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc78ee-de87-4603-b388-4edd33eadd5d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:06.806060 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc78ee-de87-4603-b388-4edd33eadd5d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:06.806345 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc78ee-de87-4603-b388-4edd33eadd5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:06.819647 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d58f26-035e-4ef9-adb4-c911bea558a1 None admin] RBAC: Authorizing `identity:get_user(user_id=94416083e41b46c798a997615dd9b5ec)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:06.821401 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d58f26-035e-4ef9-adb4-c911bea558a1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:06.823421 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 93/188] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:20:06 2022] GET /identity/v3/users/94416083e41b46c798a997615dd9b5ec => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:06.827548 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0abc78ee-de87-4603-b388-4edd33eadd5d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:06.828807 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d58f26-035e-4ef9-adb4-c911bea558a1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:06.829275 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0abc78ee-de87-4603-b388-4edd33eadd5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:06.901548 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 96/189] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:06.911387 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc78ee-de87-4603-b388-4edd33eadd5d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:06.939223 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d58f26-035e-4ef9-adb4-c911bea558a1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:06.973204 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd161de-c345-4819-b9ea-013f0be6046e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:06.973853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd161de-c345-4819-b9ea-013f0be6046e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:06.974137 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd161de-c345-4819-b9ea-013f0be6046e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:06.974411 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd161de-c345-4819-b9ea-013f0be6046e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:07.011960 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:07.031053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc78ee-de87-4603-b388-4edd33eadd5d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:07.054778 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd161de-c345-4819-b9ea-013f0be6046e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:07.056791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd161de-c345-4819-b9ea-013f0be6046e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:07.064056 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:07.071270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c295929-3e65-442d-b9ad-b6f83ded30de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:07.072086 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c295929-3e65-442d-b9ad-b6f83ded30de None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:07.072374 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c295929-3e65-442d-b9ad-b6f83ded30de None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:07.072650 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c295929-3e65-442d-b9ad-b6f83ded30de None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:07.084691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c295929-3e65-442d-b9ad-b6f83ded30de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:07.093307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2c295929-3e65-442d-b9ad-b6f83ded30de None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 12:20:07.093307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:07.093307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:07.093307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:07.093307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:07.093307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:07.093307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:07.093307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:07.093307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:07.093307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:07.093307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:07.093307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:07.093307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:07.093307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:07.093307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:07.093307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:07.094166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:07.094166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:07.094166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:07.094166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:07.094166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:07.094166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:07.094166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:07.094166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:07.094166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:07.094166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:07.094166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:07.094166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:07.094166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:07.094166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:07.094166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:07.094166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:07.095009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:07.095009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:07.095009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:07.095009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:07.095009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:07.095009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:07.095009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:07.095009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:07.095009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:07.095009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:07.095009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 12:20:07.095009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:07.095009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 97/190] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 12:20:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:07.095650 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c295929-3e65-442d-b9ad-b6f83ded30de None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:07.166467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 94/191] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:20:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:20:07.166855 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd161de-c345-4819-b9ea-013f0be6046e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:07.211623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c295929-3e65-442d-b9ad-b6f83ded30de None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:07.231790 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:07.252804 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d764e-d9e0-4c20-ab33-39d3f3aff0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:07.253597 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d764e-d9e0-4c20-ab33-39d3f3aff0c6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:07.253994 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d764e-d9e0-4c20-ab33-39d3f3aff0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:07.254270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d764e-d9e0-4c20-ab33-39d3f3aff0c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:07.258093 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d764e-d9e0-4c20-ab33-39d3f3aff0c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:07.259593 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483d764e-d9e0-4c20-ab33-39d3f3aff0c6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:07.261601 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 98/192] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:07.261997 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d764e-d9e0-4c20-ab33-39d3f3aff0c6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:07.278839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd161de-c345-4819-b9ea-013f0be6046e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:07.313743 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a7ca9-d819-484a-9ca2-6414d640945a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:07.314457 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a7ca9-d819-484a-9ca2-6414d640945a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:07.314728 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a7ca9-d819-484a-9ca2-6414d640945a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:07.315006 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4a7ca9-d819-484a-9ca2-6414d640945a None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:07.316044 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a7ca9-d819-484a-9ca2-6414d640945a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:07.316350 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a7ca9-d819-484a-9ca2-6414d640945a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:07.317848 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4a7ca9-d819-484a-9ca2-6414d640945a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:07.327354 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 95/193] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:20:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:07.328161 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a7ca9-d819-484a-9ca2-6414d640945a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:07.335200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:07.379664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d764e-d9e0-4c20-ab33-39d3f3aff0c6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:07.412792 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651555a2-32dc-4289-954d-68dcc7f4c718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:07.413651 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651555a2-32dc-4289-954d-68dcc7f4c718 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:07.413939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651555a2-32dc-4289-954d-68dcc7f4c718 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:07.414226 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651555a2-32dc-4289-954d-68dcc7f4c718 None admin] PATH_INFO: `/v3/projects/9cd49b1ddb8042ec8620180f9db712a0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:07.415466 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651555a2-32dc-4289-954d-68dcc7f4c718 None admin] RBAC: Authorizing `identity:get_project(project_id=9cd49b1ddb8042ec8620180f9db712a0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:07.416906 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651555a2-32dc-4289-954d-68dcc7f4c718 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:07.419174 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 99/194] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:07 2022] GET /identity/v3/projects/9cd49b1ddb8042ec8620180f9db712a0 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:07.420446 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6c7aa-e1f5-4547-a6c4-923ba789d420 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:07.420640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6c7aa-e1f5-4547-a6c4-923ba789d420 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:07.420926 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6c7aa-e1f5-4547-a6c4-923ba789d420 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:07.422162 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 100/195] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:07.426170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6c7aa-e1f5-4547-a6c4-923ba789d420 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:07.434105 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4a7ca9-d819-484a-9ca2-6414d640945a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:07.459052 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:07.468698 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c192a-8f3d-4503-a4af-71ebd960395f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:07.469287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c192a-8f3d-4503-a4af-71ebd960395f None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:07.469574 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c192a-8f3d-4503-a4af-71ebd960395f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:07.469854 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c192a-8f3d-4503-a4af-71ebd960395f None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:07.470603 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c192a-8f3d-4503-a4af-71ebd960395f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:07.472049 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c192a-8f3d-4503-a4af-71ebd960395f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:07.493047 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 96/196] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 12:20:07 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:20:07.493689 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c192a-8f3d-4503-a4af-71ebd960395f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:07.533127 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6c7aa-e1f5-4547-a6c4-923ba789d420 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:07.564507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:07.575717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ae136-dc0c-4352-8469-b923cc204be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:07.576623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ae136-dc0c-4352-8469-b923cc204be2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:07.576933 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ae136-dc0c-4352-8469-b923cc204be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:07.577196 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ae136-dc0c-4352-8469-b923cc204be2 None admin] PATH_INFO: `/v3/users/94416083e41b46c798a997615dd9b5ec` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:07.578566 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ae136-dc0c-4352-8469-b923cc204be2 None admin] RBAC: Authorizing `identity:get_user(user_id=94416083e41b46c798a997615dd9b5ec)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:07.580049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ae136-dc0c-4352-8469-b923cc204be2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:07.586625 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 101/197] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:20:07 2022] GET /identity/v3/users/94416083e41b46c798a997615dd9b5ec => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:07.588152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eb9fe-4782-49e4-a012-edea3d1c6ddd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:07.588501 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eb9fe-4782-49e4-a012-edea3d1c6ddd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:07.588791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eb9fe-4782-49e4-a012-edea3d1c6ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:07.607417 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9e1eb9fe-4782-49e4-a012-edea3d1c6ddd None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:07.609195 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9e1eb9fe-4782-49e4-a012-edea3d1c6ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:07.620762 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c192a-8f3d-4503-a4af-71ebd960395f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:07.661887 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eda740-d8c5-4c4f-8373-d2a8c73f09f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:07.662679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eda740-d8c5-4c4f-8373-d2a8c73f09f4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:07.662975 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eda740-d8c5-4c4f-8373-d2a8c73f09f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:07.663265 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eda740-d8c5-4c4f-8373-d2a8c73f09f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:07.664111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eda740-d8c5-4c4f-8373-d2a8c73f09f4 None admin] RBAC: Adding query filter params (role.id=3a7f4bf556344407a6d9540c89d13641, scope.project.id=9cd49b1ddb8042ec8620180f9db712a0, user.id=94416083e41b46c798a997615dd9b5ec) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:07.664430 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eda740-d8c5-4c4f-8373-d2a8c73f09f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:07.665885 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eda740-d8c5-4c4f-8373-d2a8c73f09f4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:07.672814 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 97/198] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:20:07 2022] GET /identity/v3/role_assignments?user.id=94416083e41b46c798a997615dd9b5ec&scope.project.id=9cd49b1ddb8042ec8620180f9db712a0&role.id=3a7f4bf556344407a6d9540c89d13641 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:07.673982 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eda740-d8c5-4c4f-8373-d2a8c73f09f4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:07.697306 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 102/199] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:07.723810 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1eb9fe-4782-49e4-a012-edea3d1c6ddd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:07.759686 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:07.832034 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eda740-d8c5-4c4f-8373-d2a8c73f09f4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:07.875042 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1eb9fe-4782-49e4-a012-edea3d1c6ddd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:07.882356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21117e-689f-44b3-b658-7f4495b08990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:07.883243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21117e-689f-44b3-b658-7f4495b08990 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:07.883671 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21117e-689f-44b3-b658-7f4495b08990 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:07.884120 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c21117e-689f-44b3-b658-7f4495b08990 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:07.907357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:07.923520 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998e5c7-1258-4128-87bf-f8b88ac299e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:07.924595 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998e5c7-1258-4128-87bf-f8b88ac299e5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:07.925049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998e5c7-1258-4128-87bf-f8b88ac299e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:07.925494 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7998e5c7-1258-4128-87bf-f8b88ac299e5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:07.940707 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7998e5c7-1258-4128-87bf-f8b88ac299e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:07.952053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7998e5c7-1258-4128-87bf-f8b88ac299e5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 12:20:07.952053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:07.952053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:07.952053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:07.952053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:07.952053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:07.952053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:07.952053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:07.952053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:07.952053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:07.952053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:07.952053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:07.952053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:07.952053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:07.952053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:07.952053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:07.952968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:07.952968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:07.952968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:07.952968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:07.952968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:07.952968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:07.952968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:07.952968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:07.952968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:07.952968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:07.952968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:07.952968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:07.952968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:07.952968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:07.952968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:07.952968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:07.953738 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:07.953738 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:07.953738 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:07.953738 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:07.953738 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:07.953738 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:07.953738 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:07.953738 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:07.953738 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:07.953738 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:07.953738 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 12:20:07.953738 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:07.956283 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 103/200] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:20:07.963194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998e5c7-1258-4128-87bf-f8b88ac299e5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:07.965043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c21117e-689f-44b3-b658-7f4495b08990 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:07.966852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c21117e-689f-44b3-b658-7f4495b08990 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:08.025299 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:08.089096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 98/201] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:20:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:20:08.096415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7998e5c7-1258-4128-87bf-f8b88ac299e5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:08.096718 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21117e-689f-44b3-b658-7f4495b08990 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:08.203141 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b8247-4838-4677-aa9a-8d7e50959654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:08.203822 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b8247-4838-4677-aa9a-8d7e50959654 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:08.204104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b8247-4838-4677-aa9a-8d7e50959654 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:08.204381 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b8247-4838-4677-aa9a-8d7e50959654 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:08.205145 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b8247-4838-4677-aa9a-8d7e50959654 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:08.205454 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b8247-4838-4677-aa9a-8d7e50959654 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:08.207148 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b8247-4838-4677-aa9a-8d7e50959654 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:08.224792 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 104/202] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 12:20:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:08.228965 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b8247-4838-4677-aa9a-8d7e50959654 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:08.229409 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c21117e-689f-44b3-b658-7f4495b08990 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:08.302906 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:08.353069 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b8247-4838-4677-aa9a-8d7e50959654 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:08.359691 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64489d2c-7c87-4c5f-bd74-a9b83db18621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:08.360767 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64489d2c-7c87-4c5f-bd74-a9b83db18621 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:08.361190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64489d2c-7c87-4c5f-bd74-a9b83db18621 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:08.361633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64489d2c-7c87-4c5f-bd74-a9b83db18621 None admin] PATH_INFO: `/v3/projects/9cd49b1ddb8042ec8620180f9db712a0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:08.363021 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64489d2c-7c87-4c5f-bd74-a9b83db18621 None admin] RBAC: Authorizing `identity:get_project(project_id=9cd49b1ddb8042ec8620180f9db712a0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:08.364499 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64489d2c-7c87-4c5f-bd74-a9b83db18621 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:08.366214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 99/203] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:08 2022] GET /identity/v3/projects/9cd49b1ddb8042ec8620180f9db712a0 => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:08.375440 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64489d2c-7c87-4c5f-bd74-a9b83db18621 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:08.393621 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94768b40-3ec2-49b9-b0e0-0b3f5d72b885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:08.394693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94768b40-3ec2-49b9-b0e0-0b3f5d72b885 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:08.395082 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94768b40-3ec2-49b9-b0e0-0b3f5d72b885 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:08.395461 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94768b40-3ec2-49b9-b0e0-0b3f5d72b885 None admin] PATH_INFO: `/v3/users/1399566127d74b358cd75e59fc011660` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:08.397546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94768b40-3ec2-49b9-b0e0-0b3f5d72b885 None admin] RBAC: Authorizing `identity:get_user(user_id=1399566127d74b358cd75e59fc011660)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:08.399530 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94768b40-3ec2-49b9-b0e0-0b3f5d72b885 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:08.402239 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 105/204] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:20:08 2022] GET /identity/v3/users/1399566127d74b358cd75e59fc011660 => generated 289 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:08.407403 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94768b40-3ec2-49b9-b0e0-0b3f5d72b885 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:08.409114 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:08.560686 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64489d2c-7c87-4c5f-bd74-a9b83db18621 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:08.587625 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94768b40-3ec2-49b9-b0e0-0b3f5d72b885 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:08.610799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:08.622999 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9ffd7-7250-4f20-b4e9-27b2f5e38f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:08.623668 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9ffd7-7250-4f20-b4e9-27b2f5e38f32 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:08.623951 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9ffd7-7250-4f20-b4e9-27b2f5e38f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:08.624231 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9ffd7-7250-4f20-b4e9-27b2f5e38f32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:08.625071 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9ffd7-7250-4f20-b4e9-27b2f5e38f32 None admin] RBAC: Adding query filter params (role.id=6005bd32383a4706a69108f7a2cf3ed1, scope.project.id=9cd49b1ddb8042ec8620180f9db712a0, user.id=94416083e41b46c798a997615dd9b5ec) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:08.625386 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9ffd7-7250-4f20-b4e9-27b2f5e38f32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:08.627094 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9ffd7-7250-4f20-b4e9-27b2f5e38f32 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:08.637388 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 100/205] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:20:08 2022] GET /identity/v3/role_assignments?user.id=94416083e41b46c798a997615dd9b5ec&scope.project.id=9cd49b1ddb8042ec8620180f9db712a0&role.id=6005bd32383a4706a69108f7a2cf3ed1 => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:08.644544 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbffa5d-b6fe-4ab7-9b9c-9b37e7e30364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:08.645229 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbffa5d-b6fe-4ab7-9b9c-9b37e7e30364 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:08.645519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbffa5d-b6fe-4ab7-9b9c-9b37e7e30364 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:08.645799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbffa5d-b6fe-4ab7-9b9c-9b37e7e30364 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:08.734117 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbffa5d-b6fe-4ab7-9b9c-9b37e7e30364 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:08.735680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbffa5d-b6fe-4ab7-9b9c-9b37e7e30364 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:08.848947 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 106/206] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:20:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 12:20:08.854490 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9ffd7-7250-4f20-b4e9-27b2f5e38f32 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:08.938774 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:08.961470 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9ffd7-7250-4f20-b4e9-27b2f5e38f32 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:09.022035 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b3d80-8755-442d-b300-2ca60b350e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:09.022956 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b3d80-8755-442d-b300-2ca60b350e81 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:09.023250 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b3d80-8755-442d-b300-2ca60b350e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:09.023531 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b3d80-8755-442d-b300-2ca60b350e81 None admin] PATH_INFO: `/v3/projects/a19c893c70c84649bb5406d2cf9e2708` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:09.024777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b3d80-8755-442d-b300-2ca60b350e81 None admin] RBAC: Authorizing `identity:get_project(project_id=a19c893c70c84649bb5406d2cf9e2708)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:09.026243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b3d80-8755-442d-b300-2ca60b350e81 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:09.028102 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 101/207] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:08 2022] GET /identity/v3/projects/a19c893c70c84649bb5406d2cf9e2708 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:09.034031 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbffa5d-b6fe-4ab7-9b9c-9b37e7e30364 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:09.054778 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:09.161882 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbffa5d-b6fe-4ab7-9b9c-9b37e7e30364 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:09.208011 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad5b62-f9a5-4cc5-b79a-008feef1b377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:09.208715 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad5b62-f9a5-4cc5-b79a-008feef1b377 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:09.208994 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad5b62-f9a5-4cc5-b79a-008feef1b377 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:09.209278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad5b62-f9a5-4cc5-b79a-008feef1b377 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:09.210142 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad5b62-f9a5-4cc5-b79a-008feef1b377 None admin] RBAC: Adding query filter params (role.id=c2e0da29cd7947288c717f7b4b262ef8, scope.project.id=a19c893c70c84649bb5406d2cf9e2708, user.id=1399566127d74b358cd75e59fc011660) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:09.210481 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad5b62-f9a5-4cc5-b79a-008feef1b377 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:09.211971 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad5b62-f9a5-4cc5-b79a-008feef1b377 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:09.219418 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 107/208] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:20:09 2022] GET /identity/v3/role_assignments?user.id=1399566127d74b358cd75e59fc011660&scope.project.id=a19c893c70c84649bb5406d2cf9e2708&role.id=c2e0da29cd7947288c717f7b4b262ef8 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:10.070899 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e34ca-6ea6-471f-aefd-9a302bde62be None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:10.071455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e34ca-6ea6-471f-aefd-9a302bde62be None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:10.071455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e34ca-6ea6-471f-aefd-9a302bde62be None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:10.072915 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 102/209] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:10.079548 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c809b9c-2b81-4598-9bb7-401f24d4637a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:10.080046 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c809b9c-2b81-4598-9bb7-401f24d4637a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:10.080154 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c809b9c-2b81-4598-9bb7-401f24d4637a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:10.097546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6c809b9c-2b81-4598-9bb7-401f24d4637a None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:10.098777 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6c809b9c-2b81-4598-9bb7-401f24d4637a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:10.112723 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6025dd-5063-438a-b490-906b0deb37d9 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:10.113009 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6025dd-5063-438a-b490-906b0deb37d9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:10.113291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6025dd-5063-438a-b490-906b0deb37d9 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:10.114913 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 103/210] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:10.119898 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1508b1b-0b50-478a-b7af-7ab4393e4a8e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:10.120195 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1508b1b-0b50-478a-b7af-7ab4393e4a8e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:10.121070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1508b1b-0b50-478a-b7af-7ab4393e4a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:10.137814 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a1508b1b-0b50-478a-b7af-7ab4393e4a8e None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:10.139124 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a1508b1b-0b50-478a-b7af-7ab4393e4a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:10.178760 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 108/211] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:10.184947 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db086f0-2eb8-4020-bf78-341e1e26ffee None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:10.185313 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db086f0-2eb8-4020-bf78-341e1e26ffee None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:10.185662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db086f0-2eb8-4020-bf78-341e1e26ffee None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:10.193638 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 109/212] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:20:10.193638 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4687975-5620-4c7d-9a2a-538510c6451c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:10.193638 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4687975-5620-4c7d-9a2a-538510c6451c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:10.193638 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4687975-5620-4c7d-9a2a-538510c6451c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:10.210916 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d4687975-5620-4c7d-9a2a-538510c6451c None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:10.212191 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d4687975-5620-4c7d-9a2a-538510c6451c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:10.214302 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:10.223469 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 104/213] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:20:10.224567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1508b1b-0b50-478a-b7af-7ab4393e4a8e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:10.304813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 110/214] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:20:10.305479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4687975-5620-4c7d-9a2a-538510c6451c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:10.330635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:10.360356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1508b1b-0b50-478a-b7af-7ab4393e4a8e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:10.398844 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00e880-dc66-4648-9818-9b8d56276600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:10.399762 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00e880-dc66-4648-9818-9b8d56276600 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:10.400046 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00e880-dc66-4648-9818-9b8d56276600 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:10.400324 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00e880-dc66-4648-9818-9b8d56276600 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:10.411089 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b00e880-dc66-4648-9818-9b8d56276600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:10.420077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-3b00e880-dc66-4648-9818-9b8d56276600 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:20:10.420077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:10.420077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:10.420077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:10.420077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:10.420077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:10.420077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:10.420077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:10.420077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:10.420077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:10.420077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:10.420077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:10.420077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:10.420077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:10.420077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:10.420077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:10.420970 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:10.420970 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:10.420970 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:10.420970 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:10.420970 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:10.420970 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:10.420970 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:10.420970 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:10.420970 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:10.420970 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:10.420970 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:10.420970 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:10.420970 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:10.420970 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:10.420970 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:10.420970 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:10.421750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:10.421750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:10.421750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:10.421750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:10.421750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:10.421750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:10.421750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:10.421750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:10.421750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:10.421750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:10.421750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:20:10.421750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:10.422364 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4687975-5620-4c7d-9a2a-538510c6451c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:10.422471 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 105/215] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:10.422471 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00e880-dc66-4648-9818-9b8d56276600 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:10.460615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b43f7-4015-43d9-b05d-46484468be77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:10.461414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b43f7-4015-43d9-b05d-46484468be77 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:10.461700 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b43f7-4015-43d9-b05d-46484468be77 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:10.461979 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b43f7-4015-43d9-b05d-46484468be77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:10.469199 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:10.473692 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45b43f7-4015-43d9-b05d-46484468be77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:10.481911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c45b43f7-4015-43d9-b05d-46484468be77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:20:10.481911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:10.481911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:10.481911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:10.481911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:10.481911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:10.481911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:10.481911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:10.481911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:10.481911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:10.481911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:10.481911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:10.481911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:10.481911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:10.481911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:10.481911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:10.483044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:10.483044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:10.483044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:10.483044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:10.483044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:10.483044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:10.483044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:10.483044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:10.483044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:10.483044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:10.483044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:10.483044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:10.483044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:10.483044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:10.483044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:10.483044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:10.488748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:10.488748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:10.488748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:10.488748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:10.488748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:10.488748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:10.488748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:10.488748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:10.488748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:10.488748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:10.488748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:20:10.488748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:10.488748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 111/216] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:20:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:10.488748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b43f7-4015-43d9-b05d-46484468be77 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:10.538960 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00e880-dc66-4648-9818-9b8d56276600 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:10.575458 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87825f-b6bc-4743-882d-fee30e2d6f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:10.576251 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87825f-b6bc-4743-882d-fee30e2d6f17 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:10.576534 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87825f-b6bc-4743-882d-fee30e2d6f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:10.576807 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87825f-b6bc-4743-882d-fee30e2d6f17 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:10.578053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be87825f-b6bc-4743-882d-fee30e2d6f17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:10.579615 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be87825f-b6bc-4743-882d-fee30e2d6f17 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:10.581314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 106/217] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:10.582021 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87825f-b6bc-4743-882d-fee30e2d6f17 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:10.585443 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:10.596413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b43f7-4015-43d9-b05d-46484468be77 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:10.633026 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43f749-c1d4-46d1-bb50-a9ce255ede23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:10.633808 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43f749-c1d4-46d1-bb50-a9ce255ede23 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:10.634098 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43f749-c1d4-46d1-bb50-a9ce255ede23 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:10.634378 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d43f749-c1d4-46d1-bb50-a9ce255ede23 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:10.635165 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43f749-c1d4-46d1-bb50-a9ce255ede23 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:10.635475 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43f749-c1d4-46d1-bb50-a9ce255ede23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:10.637102 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d43f749-c1d4-46d1-bb50-a9ce255ede23 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:10.645642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 112/218] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 12:20:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:10.646720 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43f749-c1d4-46d1-bb50-a9ce255ede23 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:10.693106 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87825f-b6bc-4743-882d-fee30e2d6f17 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:10.710901 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:10.728840 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b97b6e-ebce-44f6-978f-3b13146779b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:10.729513 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b97b6e-ebce-44f6-978f-3b13146779b1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:10.730336 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b97b6e-ebce-44f6-978f-3b13146779b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:10.730663 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b97b6e-ebce-44f6-978f-3b13146779b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:10.731427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b97b6e-ebce-44f6-978f-3b13146779b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:10.731734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b97b6e-ebce-44f6-978f-3b13146779b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:10.733120 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b97b6e-ebce-44f6-978f-3b13146779b1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:10.745993 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 107/219] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:10.745993 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b97b6e-ebce-44f6-978f-3b13146779b1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:10.760307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d43f749-c1d4-46d1-bb50-a9ce255ede23 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:10.798366 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8f6ef-b076-4f9d-a94f-295d0c27d04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:10.799180 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8f6ef-b076-4f9d-a94f-295d0c27d04c None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:10.799466 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8f6ef-b076-4f9d-a94f-295d0c27d04c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:10.799808 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8f6ef-b076-4f9d-a94f-295d0c27d04c None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:10.800452 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee8f6ef-b076-4f9d-a94f-295d0c27d04c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:10.801917 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee8f6ef-b076-4f9d-a94f-295d0c27d04c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:10.811178 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:10.824436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 113/220] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 12:20:10 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:20:10.825504 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8f6ef-b076-4f9d-a94f-295d0c27d04c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:10.862958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b97b6e-ebce-44f6-978f-3b13146779b1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:10.907657 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98201f-800e-4314-b1ac-c57a4a4ac084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:10.908964 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98201f-800e-4314-b1ac-c57a4a4ac084 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:10.909482 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98201f-800e-4314-b1ac-c57a4a4ac084 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:10.910059 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf98201f-800e-4314-b1ac-c57a4a4ac084 None admin] PATH_INFO: `/v3/users/5d6e0a8cea12467fbb9d6953e1dde05f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:10.927221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98201f-800e-4314-b1ac-c57a4a4ac084 None admin] RBAC: Authorizing `identity:get_user(user_id=5d6e0a8cea12467fbb9d6953e1dde05f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:10.929146 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf98201f-800e-4314-b1ac-c57a4a4ac084 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:10.931326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 108/221] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:20:10 2022] GET /identity/v3/users/5d6e0a8cea12467fbb9d6953e1dde05f => generated 297 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:10.938084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98201f-800e-4314-b1ac-c57a4a4ac084 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:10.963057 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8f6ef-b076-4f9d-a94f-295d0c27d04c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:10.991395 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:11.023948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9720066f-1a23-44c4-8ecf-dfcdab66e9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:11.024893 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720066f-1a23-44c4-8ecf-dfcdab66e9c7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:11.025179 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720066f-1a23-44c4-8ecf-dfcdab66e9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:11.025499 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720066f-1a23-44c4-8ecf-dfcdab66e9c7 None admin] PATH_INFO: `/v3/users/94416083e41b46c798a997615dd9b5ec` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:11.027708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9720066f-1a23-44c4-8ecf-dfcdab66e9c7 None admin] RBAC: Authorizing `identity:get_user(user_id=94416083e41b46c798a997615dd9b5ec)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:11.029194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9720066f-1a23-44c4-8ecf-dfcdab66e9c7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:11.032127 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 114/222] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:20:10 2022] GET /identity/v3/users/94416083e41b46c798a997615dd9b5ec => generated 283 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:11.037734 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9720066f-1a23-44c4-8ecf-dfcdab66e9c7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:11.079762 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf98201f-800e-4314-b1ac-c57a4a4ac084 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:11.110978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:11.125572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c574e8-c415-4bd9-9dc2-134cd33bbfa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:11.126174 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c574e8-c415-4bd9-9dc2-134cd33bbfa3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:11.126577 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c574e8-c415-4bd9-9dc2-134cd33bbfa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:11.126869 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c574e8-c415-4bd9-9dc2-134cd33bbfa3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:11.170462 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9720066f-1a23-44c4-8ecf-dfcdab66e9c7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:11.212826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c574e8-c415-4bd9-9dc2-134cd33bbfa3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:11.214897 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c574e8-c415-4bd9-9dc2-134cd33bbfa3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:11.226751 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b355d-ab20-4a17-8b86-53e498267004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:11.228124 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b355d-ab20-4a17-8b86-53e498267004 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:11.228483 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b355d-ab20-4a17-8b86-53e498267004 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:11.229110 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b355d-ab20-4a17-8b86-53e498267004 None admin] PATH_INFO: `/v3/projects/9cd49b1ddb8042ec8620180f9db712a0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:11.231196 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849b355d-ab20-4a17-8b86-53e498267004 None admin] RBAC: Authorizing `identity:get_project(project_id=9cd49b1ddb8042ec8620180f9db712a0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:11.233367 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849b355d-ab20-4a17-8b86-53e498267004 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:11.235989 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 115/223] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:11 2022] GET /identity/v3/projects/9cd49b1ddb8042ec8620180f9db712a0 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:11.237360 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3fe9be-901e-4c85-80b4-9f83969aaf0b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:11.237770 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3fe9be-901e-4c85-80b4-9f83969aaf0b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:11.238124 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3fe9be-901e-4c85-80b4-9f83969aaf0b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:11.240046 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 116/224] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:11.243410 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3fe9be-901e-4c85-80b4-9f83969aaf0b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:11.325255 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:11.333967 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 109/225] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:20:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 12:20:11.340490 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c09869-04ec-4c2a-b09c-e82d7d05a83f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:11.341150 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c09869-04ec-4c2a-b09c-e82d7d05a83f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:11.341632 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c09869-04ec-4c2a-b09c-e82d7d05a83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:11.372422 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-05c09869-04ec-4c2a-b09c-e82d7d05a83f None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:11.375031 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-05c09869-04ec-4c2a-b09c-e82d7d05a83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:11.386960 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3fe9be-901e-4c85-80b4-9f83969aaf0b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:11.405543 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:11.450043 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd824a7-5c77-4026-93cd-2a3645a8c00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:11.451840 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd824a7-5c77-4026-93cd-2a3645a8c00b None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:11.452138 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd824a7-5c77-4026-93cd-2a3645a8c00b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:11.452419 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd824a7-5c77-4026-93cd-2a3645a8c00b None admin] PATH_INFO: `/v3/projects/9cd49b1ddb8042ec8620180f9db712a0/users/94416083e41b46c798a997615dd9b5ec/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:11.456491 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd824a7-5c77-4026-93cd-2a3645a8c00b None admin] RBAC: Authorizing `identity:create_grant(project_id=9cd49b1ddb8042ec8620180f9db712a0, user_id=94416083e41b46c798a997615dd9b5ec, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:11.460016 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd824a7-5c77-4026-93cd-2a3645a8c00b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:11.482105 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 117/226] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 12:20:11 2022] PUT /identity/v3/projects/9cd49b1ddb8042ec8620180f9db712a0/users/94416083e41b46c798a997615dd9b5ec/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:20:11.483277 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd824a7-5c77-4026-93cd-2a3645a8c00b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:11.490835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 110/227] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:11.524907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c09869-04ec-4c2a-b09c-e82d7d05a83f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:11.685451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd824a7-5c77-4026-93cd-2a3645a8c00b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:11.700695 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:11.703629 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c09869-04ec-4c2a-b09c-e82d7d05a83f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:11.727454 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc3c97-ec2b-4106-b633-fa1924366018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:11.728708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc3c97-ec2b-4106-b633-fa1924366018 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:11.728708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc3c97-ec2b-4106-b633-fa1924366018 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:11.728970 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fc3c97-ec2b-4106-b633-fa1924366018 None admin] PATH_INFO: `/v3/projects/9cd49b1ddb8042ec8620180f9db712a0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:11.730224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc3c97-ec2b-4106-b633-fa1924366018 None admin] RBAC: Authorizing `identity:get_project(project_id=9cd49b1ddb8042ec8620180f9db712a0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:11.733797 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fc3c97-ec2b-4106-b633-fa1924366018 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:11.736451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 118/228] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:11 2022] GET /identity/v3/projects/9cd49b1ddb8042ec8620180f9db712a0 => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:11.741947 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc3c97-ec2b-4106-b633-fa1924366018 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:11.745125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21548fbb-d08d-4f22-88e1-ac90b126cac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:11.747203 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21548fbb-d08d-4f22-88e1-ac90b126cac4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:11.747636 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21548fbb-d08d-4f22-88e1-ac90b126cac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:11.748075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21548fbb-d08d-4f22-88e1-ac90b126cac4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:11.771298 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21548fbb-d08d-4f22-88e1-ac90b126cac4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:11.780307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-21548fbb-d08d-4f22-88e1-ac90b126cac4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 12:20:11.780307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:11.780307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:11.780307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:11.780307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:11.780307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:11.780307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:11.780307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:11.780307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:11.780307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:11.780307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:11.780307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:11.780307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:11.780307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:11.780307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:11.780307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:11.781249 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:11.781249 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:11.781249 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:11.781249 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:11.781249 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:11.781249 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:11.781249 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:11.781249 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:11.781249 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:11.781249 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:11.781249 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:11.781249 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:11.781249 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:11.781249 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:11.781249 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:11.781249 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:11.782036 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:11.782036 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:11.782036 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:11.782036 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:11.782036 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:11.782036 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:11.782036 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:11.782036 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:11.782036 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:11.782036 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:11.782036 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 12:20:11.782036 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:11.784536 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 111/229] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 12:20:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:20:11.799624 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21548fbb-d08d-4f22-88e1-ac90b126cac4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:11.832343 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:11.875015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fc3c97-ec2b-4106-b633-fa1924366018 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:11.919238 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ac72a-0bc0-4f32-9d86-93d398b5d24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:11.920269 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ac72a-0bc0-4f32-9d86-93d398b5d24e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:11.920772 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ac72a-0bc0-4f32-9d86-93d398b5d24e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:11.921268 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ac72a-0bc0-4f32-9d86-93d398b5d24e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:11.922307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037ac72a-0bc0-4f32-9d86-93d398b5d24e None admin] RBAC: Adding query filter params (role.id=3bf034b395b44d88b3af67da01a6ed82, scope.project.id=9cd49b1ddb8042ec8620180f9db712a0, user.id=5d6e0a8cea12467fbb9d6953e1dde05f) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:11.922837 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037ac72a-0bc0-4f32-9d86-93d398b5d24e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:11.924484 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037ac72a-0bc0-4f32-9d86-93d398b5d24e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:11.939195 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 119/230] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:20:11 2022] GET /identity/v3/role_assignments?user.id=5d6e0a8cea12467fbb9d6953e1dde05f&scope.project.id=9cd49b1ddb8042ec8620180f9db712a0&role.id=3bf034b395b44d88b3af67da01a6ed82 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:11.959136 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21548fbb-d08d-4f22-88e1-ac90b126cac4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:12.019324 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad89574-bc0a-4ef5-b6f9-a9afffc33694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:12.020289 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad89574-bc0a-4ef5-b6f9-a9afffc33694 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:12.020687 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad89574-bc0a-4ef5-b6f9-a9afffc33694 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:12.021342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad89574-bc0a-4ef5-b6f9-a9afffc33694 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:12.022502 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad89574-bc0a-4ef5-b6f9-a9afffc33694 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:12.022960 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad89574-bc0a-4ef5-b6f9-a9afffc33694 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:12.025102 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad89574-bc0a-4ef5-b6f9-a9afffc33694 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:12.044668 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 112/231] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:20:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:12.052407 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ac72a-0bc0-4f32-9d86-93d398b5d24e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:12.119693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:12.197179 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ac72a-0bc0-4f32-9d86-93d398b5d24e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:12.241894 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64bd1a-4f0d-4f41-b6c9-16d0d591210b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:12.242877 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64bd1a-4f0d-4f41-b6c9-16d0d591210b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:12.243173 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64bd1a-4f0d-4f41-b6c9-16d0d591210b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:12.243454 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64bd1a-4f0d-4f41-b6c9-16d0d591210b None admin] PATH_INFO: `/v3/users/94416083e41b46c798a997615dd9b5ec` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:12.244672 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce64bd1a-4f0d-4f41-b6c9-16d0d591210b None admin] RBAC: Authorizing `identity:get_user(user_id=94416083e41b46c798a997615dd9b5ec)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:12.246964 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce64bd1a-4f0d-4f41-b6c9-16d0d591210b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:12.249001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 120/232] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:20:12 2022] GET /identity/v3/users/94416083e41b46c798a997615dd9b5ec => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:12.255536 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad89574-bc0a-4ef5-b6f9-a9afffc33694 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:12.402696 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:12.450187 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad89574-bc0a-4ef5-b6f9-a9afffc33694 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:12.509786 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5bfc07-1811-428e-8b84-2fcafcb96735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:12.510689 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5bfc07-1811-428e-8b84-2fcafcb96735 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:12.511024 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5bfc07-1811-428e-8b84-2fcafcb96735 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:12.511344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5bfc07-1811-428e-8b84-2fcafcb96735 None admin] PATH_INFO: `/v3/projects/9cd49b1ddb8042ec8620180f9db712a0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:12.515157 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5bfc07-1811-428e-8b84-2fcafcb96735 None admin] RBAC: Authorizing `identity:get_project(project_id=9cd49b1ddb8042ec8620180f9db712a0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:12.516626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5bfc07-1811-428e-8b84-2fcafcb96735 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:12.518344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 113/233] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:12 2022] GET /identity/v3/projects/9cd49b1ddb8042ec8620180f9db712a0 => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:12.525915 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64bd1a-4f0d-4f41-b6c9-16d0d591210b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:12.635836 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64bd1a-4f0d-4f41-b6c9-16d0d591210b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:12.650202 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:12.686088 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06479f9b-ff74-4a1f-9335-62b79d276f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:12.687051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06479f9b-ff74-4a1f-9335-62b79d276f85 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:12.687340 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06479f9b-ff74-4a1f-9335-62b79d276f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:12.687620 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06479f9b-ff74-4a1f-9335-62b79d276f85 None admin] PATH_INFO: `/v3/projects/9cd49b1ddb8042ec8620180f9db712a0/users/94416083e41b46c798a997615dd9b5ec/roles/6005bd32383a4706a69108f7a2cf3ed1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:12.689923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06479f9b-ff74-4a1f-9335-62b79d276f85 None admin] RBAC: Authorizing `identity:create_grant(project_id=9cd49b1ddb8042ec8620180f9db712a0, user_id=94416083e41b46c798a997615dd9b5ec, role_id=6005bd32383a4706a69108f7a2cf3ed1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:12.691438 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06479f9b-ff74-4a1f-9335-62b79d276f85 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:12.708320 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 121/234] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 12:20:12 2022] PUT /identity/v3/projects/9cd49b1ddb8042ec8620180f9db712a0/users/94416083e41b46c798a997615dd9b5ec/roles/6005bd32383a4706a69108f7a2cf3ed1 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:20:13.312898 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e78c34-8a80-4ea1-9d03-0ec2379cdd0d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:13.313411 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e78c34-8a80-4ea1-9d03-0ec2379cdd0d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:13.313668 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e78c34-8a80-4ea1-9d03-0ec2379cdd0d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:13.316275 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 114/235] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:13.321631 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd32112-c07f-4fa1-afb4-92416815791a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:13.322726 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd32112-c07f-4fa1-afb4-92416815791a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:13.323387 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd32112-c07f-4fa1-afb4-92416815791a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:13.355525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bfd32112-c07f-4fa1-afb4-92416815791a None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:13.357731 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bfd32112-c07f-4fa1-afb4-92416815791a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:13.477308 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 122/236] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:13.522224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e78c34-8a80-4ea1-9d03-0ec2379cdd0d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:13.674172 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e78c34-8a80-4ea1-9d03-0ec2379cdd0d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:13.721652 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb02aed-1cc7-4c7c-967b-a5bb83be55f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:13.722472 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb02aed-1cc7-4c7c-967b-a5bb83be55f3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:13.722764 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb02aed-1cc7-4c7c-967b-a5bb83be55f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:13.723046 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb02aed-1cc7-4c7c-967b-a5bb83be55f3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:13.742117 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb02aed-1cc7-4c7c-967b-a5bb83be55f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:13.750040 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-3eb02aed-1cc7-4c7c-967b-a5bb83be55f3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 12:20:13.750040 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:13.750040 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:13.750040 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:13.750040 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:13.750040 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:13.750040 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:13.750040 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:13.750040 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:13.750040 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:13.750040 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:13.750040 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:13.750040 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:13.750040 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:13.750040 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:13.750040 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:13.750953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:13.750953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:13.750953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:13.750953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:13.750953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:13.750953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:13.750953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:13.750953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:13.750953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:13.750953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:13.750953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:13.750953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:13.750953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:13.750953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:13.750953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:13.750953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:13.751727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:13.751727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:13.751727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:13.751727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:13.751727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:13.751727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:13.751727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:13.751727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:13.751727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:13.751727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:13.751727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 12:20:13.751727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:13.751727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 115/237] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:13.758887 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd32112-c07f-4fa1-afb4-92416815791a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:13.917292 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd32112-c07f-4fa1-afb4-92416815791a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:13.918943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:13.967229 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afd5a6-fa67-4fd5-bba7-63f0e3d3f793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:13.968249 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afd5a6-fa67-4fd5-bba7-63f0e3d3f793 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:13.968750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afd5a6-fa67-4fd5-bba7-63f0e3d3f793 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:13.969237 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afd5a6-fa67-4fd5-bba7-63f0e3d3f793 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:13.970185 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2afd5a6-fa67-4fd5-bba7-63f0e3d3f793 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:13.970717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2afd5a6-fa67-4fd5-bba7-63f0e3d3f793 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:13.972283 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2afd5a6-fa67-4fd5-bba7-63f0e3d3f793 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:13.982271 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 123/238] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 12:20:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:13.998460 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb02aed-1cc7-4c7c-967b-a5bb83be55f3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:14.005330 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:14.149436 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb02aed-1cc7-4c7c-967b-a5bb83be55f3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:14.192734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95607f86-f64b-46b7-91a9-c9d890a90836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:14.193615 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95607f86-f64b-46b7-91a9-c9d890a90836 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:14.193900 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95607f86-f64b-46b7-91a9-c9d890a90836 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:14.194179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95607f86-f64b-46b7-91a9-c9d890a90836 None admin] PATH_INFO: `/v3/users/f73e200499404c46a96fee7624386736` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:14.213786 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69357d4-481f-4261-80c1-04c263bf6fb4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:14.214522 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69357d4-481f-4261-80c1-04c263bf6fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:14.215232 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69357d4-481f-4261-80c1-04c263bf6fb4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:14.215534 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95607f86-f64b-46b7-91a9-c9d890a90836 None admin] RBAC: Authorizing `identity:get_user(user_id=f73e200499404c46a96fee7624386736)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:14.217021 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95607f86-f64b-46b7-91a9-c9d890a90836 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:14.217218 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 124/239] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:14.220527 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 116/240] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:20:13 2022] GET /identity/v3/users/f73e200499404c46a96fee7624386736 => generated 297 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:14.225502 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69357d4-481f-4261-80c1-04c263bf6fb4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:14.228856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82175d9-7d99-4eac-a918-9da4a6d06df9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:14.229145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82175d9-7d99-4eac-a918-9da4a6d06df9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:14.229426 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82175d9-7d99-4eac-a918-9da4a6d06df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:14.265244 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f82175d9-7d99-4eac-a918-9da4a6d06df9 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:14.267828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f82175d9-7d99-4eac-a918-9da4a6d06df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:14.353981 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:14.369303 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 117/241] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:20:14.372097 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9340d6-52dc-42c7-ba89-aa9f1ba9c341 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:14.372384 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9340d6-52dc-42c7-ba89-aa9f1ba9c341 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:14.372666 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9340d6-52dc-42c7-ba89-aa9f1ba9c341 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:14.373995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 118/242] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:14.375285 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac4390-7754-45d6-8097-81d519fccff0 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:14.375719 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac4390-7754-45d6-8097-81d519fccff0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:14.376139 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac4390-7754-45d6-8097-81d519fccff0 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:14.377756 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 119/243] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:14.383119 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b30b0f-5202-4b94-8159-4dbe62bea4e3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:14.383299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69357d4-481f-4261-80c1-04c263bf6fb4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:14.384071 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b30b0f-5202-4b94-8159-4dbe62bea4e3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:14.384508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b30b0f-5202-4b94-8159-4dbe62bea4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:14.405666 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a0b30b0f-5202-4b94-8159-4dbe62bea4e3 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:14.407144 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a0b30b0f-5202-4b94-8159-4dbe62bea4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:14.422648 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f425e7-b2a7-4f80-abe8-83d4fc56246f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:14.423310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f425e7-b2a7-4f80-abe8-83d4fc56246f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:14.423602 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f425e7-b2a7-4f80-abe8-83d4fc56246f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:14.423889 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f425e7-b2a7-4f80-abe8-83d4fc56246f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:14.455567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:14.486518 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 120/244] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:14.494008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f425e7-b2a7-4f80-abe8-83d4fc56246f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:14.494278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff53f89-533c-4262-8cf5-c6705e1c496d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:14.494595 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff53f89-533c-4262-8cf5-c6705e1c496d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:14.494877 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff53f89-533c-4262-8cf5-c6705e1c496d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:14.495716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f425e7-b2a7-4f80-abe8-83d4fc56246f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:14.519086 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5ff53f89-533c-4262-8cf5-c6705e1c496d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:14.520561 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5ff53f89-533c-4262-8cf5-c6705e1c496d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:14.598357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 121/245] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:14.599066 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff53f89-533c-4262-8cf5-c6705e1c496d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:14.623971 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 125/246] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:20:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:20:14.625807 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f425e7-b2a7-4f80-abe8-83d4fc56246f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:14.657487 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:14.710370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:14.714005 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff53f89-533c-4262-8cf5-c6705e1c496d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:14.751167 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d1460b-20ed-4b85-b8a6-71924eff1f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:14.751379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f425e7-b2a7-4f80-abe8-83d4fc56246f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:14.751973 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1460b-20ed-4b85-b8a6-71924eff1f9f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:14.752260 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1460b-20ed-4b85-b8a6-71924eff1f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:14.752531 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d1460b-20ed-4b85-b8a6-71924eff1f9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:14.765513 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d1460b-20ed-4b85-b8a6-71924eff1f9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:14.777556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b3d1460b-20ed-4b85-b8a6-71924eff1f9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:20:14.777556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:14.777556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:14.777556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:14.777556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:14.777556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:14.777556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:14.777556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:14.777556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:14.777556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:14.777556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:14.777556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:14.777556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:14.777556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:14.777556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:14.777556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:14.778570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:14.778570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:14.778570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:14.778570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:14.778570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:14.778570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:14.778570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:14.778570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:14.778570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:14.778570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:14.778570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:14.778570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:14.778570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:14.778570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:14.778570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:14.778570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:14.785849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:14.785849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:14.785849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:14.785849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:14.785849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:14.785849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:14.785849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:14.785849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:14.785849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:14.785849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:14.785849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:20:14.785849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:14.785849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 122/247] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:20:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:14.785849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d1460b-20ed-4b85-b8a6-71924eff1f9f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:14.803263 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e7e9a7-7421-4e68-b071-a2e1851023b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:14.804072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7e9a7-7421-4e68-b071-a2e1851023b0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:14.804360 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7e9a7-7421-4e68-b071-a2e1851023b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:14.804640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e7e9a7-7421-4e68-b071-a2e1851023b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:14.805933 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e7e9a7-7421-4e68-b071-a2e1851023b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:14.808468 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e7e9a7-7421-4e68-b071-a2e1851023b0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:14.814642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 126/248] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:14.815530 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e7e9a7-7421-4e68-b071-a2e1851023b0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:14.921116 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d1460b-20ed-4b85-b8a6-71924eff1f9f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:14.936450 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e7e9a7-7421-4e68-b071-a2e1851023b0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:14.940366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:14.969763 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b086362-7853-4e31-b655-a34bcd777ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:14.970603 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b086362-7853-4e31-b655-a34bcd777ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:14.971009 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b086362-7853-4e31-b655-a34bcd777ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:14.971352 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b086362-7853-4e31-b655-a34bcd777ef9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:14.977075 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:14.983967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba41f7-8093-464f-a6e5-600c0c9edd61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:14.984708 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b086362-7853-4e31-b655-a34bcd777ef9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:14.984858 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba41f7-8093-464f-a6e5-600c0c9edd61 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:14.985356 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba41f7-8093-464f-a6e5-600c0c9edd61 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:14.985876 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fba41f7-8093-464f-a6e5-600c0c9edd61 None admin] PATH_INFO: `/v3/projects/9cd49b1ddb8042ec8620180f9db712a0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:14.987130 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fba41f7-8093-464f-a6e5-600c0c9edd61 None admin] RBAC: Authorizing `identity:get_project(project_id=9cd49b1ddb8042ec8620180f9db712a0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:14.988801 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fba41f7-8093-464f-a6e5-600c0c9edd61 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:14.991582 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 127/249] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:14 2022] GET /identity/v3/projects/9cd49b1ddb8042ec8620180f9db712a0 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:14.992367 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba41f7-8093-464f-a6e5-600c0c9edd61 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:14.999852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-2b086362-7853-4e31-b655-a34bcd777ef9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:20:14.999852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:14.999852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:14.999852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:14.999852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:14.999852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:14.999852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:14.999852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:14.999852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:14.999852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:14.999852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:14.999852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:14.999852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:14.999852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:14.999852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:14.999852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:15.000816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:15.000816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:15.000816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:15.000816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:15.000816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:15.000816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:15.000816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:15.000816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:15.000816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:15.000816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:15.000816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:15.000816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:15.000816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:15.000816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:15.000816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:15.000816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:15.002222 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:15.002222 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:15.002222 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:15.002222 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:15.002222 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:15.002222 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:15.002222 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:15.002222 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:15.002222 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:15.002222 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:15.002222 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:20:15.002222 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:15.002222 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 123/250] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:15.002222 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b086362-7853-4e31-b655-a34bcd777ef9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:15.124988 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fba41f7-8093-464f-a6e5-600c0c9edd61 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:15.145438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b086362-7853-4e31-b655-a34bcd777ef9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:15.171870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404b5ba-7652-4d90-99e9-02fa835555e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:15.173056 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404b5ba-7652-4d90-99e9-02fa835555e3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:15.174076 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404b5ba-7652-4d90-99e9-02fa835555e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:15.175256 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8404b5ba-7652-4d90-99e9-02fa835555e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:15.176484 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8404b5ba-7652-4d90-99e9-02fa835555e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:15.177667 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8404b5ba-7652-4d90-99e9-02fa835555e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:15.180186 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8404b5ba-7652-4d90-99e9-02fa835555e3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:15.191215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 128/251] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:15.192788 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404b5ba-7652-4d90-99e9-02fa835555e3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:15.200385 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838881a3-62d1-45d9-98f1-c9862e78bd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:15.201102 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838881a3-62d1-45d9-98f1-c9862e78bd3d None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:15.201386 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838881a3-62d1-45d9-98f1-c9862e78bd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:15.201714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838881a3-62d1-45d9-98f1-c9862e78bd3d None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:15.202475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838881a3-62d1-45d9-98f1-c9862e78bd3d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:15.204039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838881a3-62d1-45d9-98f1-c9862e78bd3d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:15.233703 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 124/252] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 12:20:15 2022] POST /identity/v3/users => generated 293 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:20:15.244672 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838881a3-62d1-45d9-98f1-c9862e78bd3d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:15.287013 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:15.289557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:15.369837 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8404b5ba-7652-4d90-99e9-02fa835555e3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:15.405131 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838881a3-62d1-45d9-98f1-c9862e78bd3d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:15.413027 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d0af7-2695-4b29-9514-493257912074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:15.413873 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d0af7-2695-4b29-9514-493257912074 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:15.414209 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d0af7-2695-4b29-9514-493257912074 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:15.414515 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d0af7-2695-4b29-9514-493257912074 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:15.415372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d0af7-2695-4b29-9514-493257912074 None admin] RBAC: Adding query filter params (role.id=c2e0da29cd7947288c717f7b4b262ef8, scope.project.id=9cd49b1ddb8042ec8620180f9db712a0, user.id=f73e200499404c46a96fee7624386736) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:15.415809 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d0af7-2695-4b29-9514-493257912074 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:15.417813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d0af7-2695-4b29-9514-493257912074 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:15.425253 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 129/253] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:20:15 2022] GET /identity/v3/role_assignments?user.id=f73e200499404c46a96fee7624386736&scope.project.id=9cd49b1ddb8042ec8620180f9db712a0&role.id=c2e0da29cd7947288c717f7b4b262ef8 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:15.430632 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d0af7-2695-4b29-9514-493257912074 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:15.457212 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a15863-1f53-45aa-834c-09a32cfc2e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:15.458044 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a15863-1f53-45aa-834c-09a32cfc2e98 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:15.458490 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a15863-1f53-45aa-834c-09a32cfc2e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:15.458779 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a15863-1f53-45aa-834c-09a32cfc2e98 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:15.459574 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a15863-1f53-45aa-834c-09a32cfc2e98 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:15.460083 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a15863-1f53-45aa-834c-09a32cfc2e98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:15.464526 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a15863-1f53-45aa-834c-09a32cfc2e98 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:15.481219 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 125/254] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 12:20:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:15.484411 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771e90f-a5ef-4328-ab23-8c085beddc0f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:15.484411 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771e90f-a5ef-4328-ab23-8c085beddc0f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:15.484411 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771e90f-a5ef-4328-ab23-8c085beddc0f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:15.487540 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 126/255] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:15.488349 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771e90f-a5ef-4328-ab23-8c085beddc0f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:15.589031 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d0af7-2695-4b29-9514-493257912074 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:15.624000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:15.633026 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2573fb-2306-48de-b45e-98eee406ae0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:15.633986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2573fb-2306-48de-b45e-98eee406ae0c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:15.634219 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2573fb-2306-48de-b45e-98eee406ae0c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:15.634530 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2573fb-2306-48de-b45e-98eee406ae0c None admin] PATH_INFO: `/v3/users/94416083e41b46c798a997615dd9b5ec` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:15.635774 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2573fb-2306-48de-b45e-98eee406ae0c None admin] RBAC: Authorizing `identity:get_user(user_id=94416083e41b46c798a997615dd9b5ec)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:15.638258 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2573fb-2306-48de-b45e-98eee406ae0c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:15.640159 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 130/256] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:20:15 2022] GET /identity/v3/users/94416083e41b46c798a997615dd9b5ec => generated 283 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:15.641210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6af0b-582e-43eb-abb9-5a8ba46adcfd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:15.641504 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6af0b-582e-43eb-abb9-5a8ba46adcfd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:15.641787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6af0b-582e-43eb-abb9-5a8ba46adcfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:15.653772 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:15.667973 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-18e6af0b-582e-43eb-abb9-5a8ba46adcfd None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:15.669457 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-18e6af0b-582e-43eb-abb9-5a8ba46adcfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:15.672461 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771e90f-a5ef-4328-ab23-8c085beddc0f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:15.719193 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a563c-5c9d-4ada-823b-15ee09289d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:15.720078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a563c-5c9d-4ada-823b-15ee09289d10 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:15.720359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a563c-5c9d-4ada-823b-15ee09289d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:15.720647 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a563c-5c9d-4ada-823b-15ee09289d10 None admin] PATH_INFO: `/v3/users/5d6e0a8cea12467fbb9d6953e1dde05f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:15.721853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a563c-5c9d-4ada-823b-15ee09289d10 None admin] RBAC: Authorizing `identity:get_user(user_id=5d6e0a8cea12467fbb9d6953e1dde05f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:15.723346 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a563c-5c9d-4ada-823b-15ee09289d10 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:15.724980 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 127/257] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:20:15 2022] GET /identity/v3/users/5d6e0a8cea12467fbb9d6953e1dde05f => generated 297 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:15.725754 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a563c-5c9d-4ada-823b-15ee09289d10 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:15.783987 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 131/258] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:15.784665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6af0b-582e-43eb-abb9-5a8ba46adcfd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:15.910779 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a563c-5c9d-4ada-823b-15ee09289d10 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:15.924816 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6af0b-582e-43eb-abb9-5a8ba46adcfd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:15.954371 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107b0c0-2d0a-44ce-be0c-5df4ae758343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:15.955291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107b0c0-2d0a-44ce-be0c-5df4ae758343 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:15.955576 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107b0c0-2d0a-44ce-be0c-5df4ae758343 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:15.955860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107b0c0-2d0a-44ce-be0c-5df4ae758343 None admin] PATH_INFO: `/v3/projects/9cd49b1ddb8042ec8620180f9db712a0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:15.957055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c107b0c0-2d0a-44ce-be0c-5df4ae758343 None admin] RBAC: Authorizing `identity:get_project(project_id=9cd49b1ddb8042ec8620180f9db712a0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:15.958552 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c107b0c0-2d0a-44ce-be0c-5df4ae758343 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:15.960255 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 132/259] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:15 2022] GET /identity/v3/projects/9cd49b1ddb8042ec8620180f9db712a0 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:15.961017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107b0c0-2d0a-44ce-be0c-5df4ae758343 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:15.974836 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:16.017595 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082186b1-77ff-4e10-96e9-9d9e5433df87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:16.018312 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082186b1-77ff-4e10-96e9-9d9e5433df87 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:16.018629 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082186b1-77ff-4e10-96e9-9d9e5433df87 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:16.018916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082186b1-77ff-4e10-96e9-9d9e5433df87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:16.082819 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:16.113388 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107b0c0-2d0a-44ce-be0c-5df4ae758343 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:16.154288 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082186b1-77ff-4e10-96e9-9d9e5433df87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:16.156554 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082186b1-77ff-4e10-96e9-9d9e5433df87 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:16.159502 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2171b2d-1c3a-4d8b-9e4b-bd5a386d7464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:16.160333 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2171b2d-1c3a-4d8b-9e4b-bd5a386d7464 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:16.160587 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2171b2d-1c3a-4d8b-9e4b-bd5a386d7464 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:16.160913 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2171b2d-1c3a-4d8b-9e4b-bd5a386d7464 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:16.181165 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2171b2d-1c3a-4d8b-9e4b-bd5a386d7464 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:16.199588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b2171b2d-1c3a-4d8b-9e4b-bd5a386d7464 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 12:20:16.199588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:16.199588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:16.199588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:16.199588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:16.199588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:16.199588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:16.199588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:16.199588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:16.199588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:16.199588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:16.199588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:16.199588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:16.199588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:16.199588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:16.199588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:16.200787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:16.200787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:16.200787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:16.200787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:16.200787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:16.200787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:16.200787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:16.200787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:16.200787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:16.200787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:16.200787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:16.200787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:16.200787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:16.200787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:16.200787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:16.200787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:16.201759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:16.201759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:16.201759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:16.201759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:16.201759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:16.201759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:16.201759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:16.201759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:16.201759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:16.201759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:16.201759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 12:20:16.201759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:16.201759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 133/260] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 12:20:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:16.201759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2171b2d-1c3a-4d8b-9e4b-bd5a386d7464 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:16.303673 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 128/261] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:20:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:20:16.304482 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082186b1-77ff-4e10-96e9-9d9e5433df87 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:16.370537 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:16.373131 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:16.450105 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082186b1-77ff-4e10-96e9-9d9e5433df87 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:16.470388 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2171b2d-1c3a-4d8b-9e4b-bd5a386d7464 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:16.516279 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ade678-0656-4e02-8dae-48b6a8a689ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:16.516954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ade678-0656-4e02-8dae-48b6a8a689ee None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:16.517240 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ade678-0656-4e02-8dae-48b6a8a689ee None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:16.517524 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ade678-0656-4e02-8dae-48b6a8a689ee None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:16.518272 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ade678-0656-4e02-8dae-48b6a8a689ee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:16.518608 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ade678-0656-4e02-8dae-48b6a8a689ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:16.520010 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ade678-0656-4e02-8dae-48b6a8a689ee None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:16.522045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c995df3d-2389-4d55-b34a-d3aac049d32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:16.522962 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995df3d-2389-4d55-b34a-d3aac049d32e None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:16.523245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995df3d-2389-4d55-b34a-d3aac049d32e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:16.523520 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995df3d-2389-4d55-b34a-d3aac049d32e None admin] PATH_INFO: `/v3/projects/9cd49b1ddb8042ec8620180f9db712a0/users/5d6e0a8cea12467fbb9d6953e1dde05f/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:16.529466 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c995df3d-2389-4d55-b34a-d3aac049d32e None admin] RBAC: Authorizing `identity:create_grant(project_id=9cd49b1ddb8042ec8620180f9db712a0, user_id=5d6e0a8cea12467fbb9d6953e1dde05f, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:16.530942 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c995df3d-2389-4d55-b34a-d3aac049d32e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:16.536565 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 129/262] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:20:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:16.537902 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ade678-0656-4e02-8dae-48b6a8a689ee None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:16.550491 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 134/263] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 12:20:16 2022] PUT /identity/v3/projects/9cd49b1ddb8042ec8620180f9db712a0/users/5d6e0a8cea12467fbb9d6953e1dde05f/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:20:16.551221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c995df3d-2389-4d55-b34a-d3aac049d32e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:16.744217 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ade678-0656-4e02-8dae-48b6a8a689ee None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:16.748396 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c995df3d-2389-4d55-b34a-d3aac049d32e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:16.770791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:16.787490 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:16.817895 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e7e44-77ac-4702-939d-8905c7d5e0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:16.818840 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e7e44-77ac-4702-939d-8905c7d5e0d8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:16.819127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e7e44-77ac-4702-939d-8905c7d5e0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:16.819448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e7e44-77ac-4702-939d-8905c7d5e0d8 None admin] PATH_INFO: `/v3/projects/9cd49b1ddb8042ec8620180f9db712a0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:16.821388 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0e7e44-77ac-4702-939d-8905c7d5e0d8 None admin] RBAC: Authorizing `identity:get_project(project_id=9cd49b1ddb8042ec8620180f9db712a0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:16.822951 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0e7e44-77ac-4702-939d-8905c7d5e0d8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:16.825813 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 130/264] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:16 2022] GET /identity/v3/projects/9cd49b1ddb8042ec8620180f9db712a0 => generated 302 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:16.826733 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d825c7-0abe-4db1-9d5c-e1e37e2ac508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:16.827699 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d825c7-0abe-4db1-9d5c-e1e37e2ac508 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:16.827990 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d825c7-0abe-4db1-9d5c-e1e37e2ac508 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:16.828265 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d825c7-0abe-4db1-9d5c-e1e37e2ac508 None admin] PATH_INFO: `/v3/users/94416083e41b46c798a997615dd9b5ec` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:16.829675 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d825c7-0abe-4db1-9d5c-e1e37e2ac508 None admin] RBAC: Authorizing `identity:get_user(user_id=94416083e41b46c798a997615dd9b5ec)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:16.831426 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d825c7-0abe-4db1-9d5c-e1e37e2ac508 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:16.833284 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 135/265] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:20:16 2022] GET /identity/v3/users/94416083e41b46c798a997615dd9b5ec => generated 283 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:16.834482 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e7e44-77ac-4702-939d-8905c7d5e0d8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:16.839493 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d825c7-0abe-4db1-9d5c-e1e37e2ac508 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:16.986378 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e7e44-77ac-4702-939d-8905c7d5e0d8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:17.031510 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd8f619-2e8a-4b76-8a52-60be2cf180c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:17.033580 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8f619-2e8a-4b76-8a52-60be2cf180c8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:17.033580 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8f619-2e8a-4b76-8a52-60be2cf180c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:17.033580 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8f619-2e8a-4b76-8a52-60be2cf180c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:17.034219 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd8f619-2e8a-4b76-8a52-60be2cf180c8 None admin] RBAC: Adding query filter params (role.id=3a7f4bf556344407a6d9540c89d13641, scope.project.id=9cd49b1ddb8042ec8620180f9db712a0, user.id=94416083e41b46c798a997615dd9b5ec) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:17.036178 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd8f619-2e8a-4b76-8a52-60be2cf180c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:17.036291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd8f619-2e8a-4b76-8a52-60be2cf180c8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:17.043742 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 131/266] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:20:16 2022] GET /identity/v3/role_assignments?user.id=94416083e41b46c798a997615dd9b5ec&scope.project.id=9cd49b1ddb8042ec8620180f9db712a0&role.id=3a7f4bf556344407a6d9540c89d13641 => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:17.054152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d825c7-0abe-4db1-9d5c-e1e37e2ac508 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:17.118901 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20410943-7595-4334-8aa4-067f5d75b145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:17.119970 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20410943-7595-4334-8aa4-067f5d75b145 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:17.120987 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20410943-7595-4334-8aa4-067f5d75b145 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:17.121463 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20410943-7595-4334-8aa4-067f5d75b145 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:17.200915 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:17.230830 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20410943-7595-4334-8aa4-067f5d75b145 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:17.232616 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20410943-7595-4334-8aa4-067f5d75b145 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:17.339430 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 136/267] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:20:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 12:20:17.351796 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd8f619-2e8a-4b76-8a52-60be2cf180c8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:17.411875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:17.460861 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd8f619-2e8a-4b76-8a52-60be2cf180c8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:17.498677 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3fe52d-86c2-4d9f-a673-22c85544e6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:17.499549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fe52d-86c2-4d9f-a673-22c85544e6eb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:17.499832 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fe52d-86c2-4d9f-a673-22c85544e6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:17.500110 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fe52d-86c2-4d9f-a673-22c85544e6eb None admin] PATH_INFO: `/v3/projects/9cd49b1ddb8042ec8620180f9db712a0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:17.501309 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3fe52d-86c2-4d9f-a673-22c85544e6eb None admin] RBAC: Authorizing `identity:get_project(project_id=9cd49b1ddb8042ec8620180f9db712a0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:17.502796 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3fe52d-86c2-4d9f-a673-22c85544e6eb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:17.504822 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 132/268] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:17 2022] GET /identity/v3/projects/9cd49b1ddb8042ec8620180f9db712a0 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:17.510440 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20410943-7595-4334-8aa4-067f5d75b145 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:17.594156 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:17.613735 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20410943-7595-4334-8aa4-067f5d75b145 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:17.650313 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490304d-14a5-4f54-af9a-57daedd353c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:17.651006 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490304d-14a5-4f54-af9a-57daedd353c6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:17.651291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490304d-14a5-4f54-af9a-57daedd353c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:17.652142 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490304d-14a5-4f54-af9a-57daedd353c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:17.653117 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3490304d-14a5-4f54-af9a-57daedd353c6 None admin] RBAC: Adding query filter params (role.id=6005bd32383a4706a69108f7a2cf3ed1, scope.project.id=9cd49b1ddb8042ec8620180f9db712a0, user.id=94416083e41b46c798a997615dd9b5ec) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:17.653429 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3490304d-14a5-4f54-af9a-57daedd353c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:17.654900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3490304d-14a5-4f54-af9a-57daedd353c6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:17.661584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 137/269] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:20:17 2022] GET /identity/v3/role_assignments?user.id=94416083e41b46c798a997615dd9b5ec&scope.project.id=9cd49b1ddb8042ec8620180f9db712a0&role.id=6005bd32383a4706a69108f7a2cf3ed1 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:18.044057 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbae0bb-a157-4a9b-b134-7d3b64336154 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:18.044448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbae0bb-a157-4a9b-b134-7d3b64336154 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:18.044620 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbae0bb-a157-4a9b-b134-7d3b64336154 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:18.046297 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 133/270] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:18.052718 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1beb9f3-da85-4f8d-8160-03ce504c6a27 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:18.053104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1beb9f3-da85-4f8d-8160-03ce504c6a27 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:18.053376 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1beb9f3-da85-4f8d-8160-03ce504c6a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:18.071162 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c1beb9f3-da85-4f8d-8160-03ce504c6a27 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:18.072343 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c1beb9f3-da85-4f8d-8160-03ce504c6a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:18.146987 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 138/271] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:18.163073 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b7b76-1142-4015-be97-adcb748c3080 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:18.163368 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b7b76-1142-4015-be97-adcb748c3080 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:18.163650 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b7b76-1142-4015-be97-adcb748c3080 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:18.165294 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 134/272] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:18.170474 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066f4e8-34fe-40db-81aa-68410f84ca82 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:18.170748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066f4e8-34fe-40db-81aa-68410f84ca82 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:18.171028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066f4e8-34fe-40db-81aa-68410f84ca82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:18.180153 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b7b76-1142-4015-be97-adcb748c3080 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:18.190789 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a066f4e8-34fe-40db-81aa-68410f84ca82 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:18.194504 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a066f4e8-34fe-40db-81aa-68410f84ca82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:18.271463 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 139/273] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:18.273069 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e6ac5-2354-4a31-a874-24daa40b5705 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:18.273501 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e6ac5-2354-4a31-a874-24daa40b5705 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:18.273943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7e6ac5-2354-4a31-a874-24daa40b5705 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:18.275610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 140/274] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:18.284998 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453f7ca-3ddc-47dd-9a28-bcf8e0d1f577 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:18.285394 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453f7ca-3ddc-47dd-9a28-bcf8e0d1f577 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:18.286240 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453f7ca-3ddc-47dd-9a28-bcf8e0d1f577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:18.300672 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:18.307364 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b453f7ca-3ddc-47dd-9a28-bcf8e0d1f577 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:18.309152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b453f7ca-3ddc-47dd-9a28-bcf8e0d1f577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:18.311584 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b7b76-1142-4015-be97-adcb748c3080 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:18.332685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:18.347666 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77daab0b-bb6b-4f6d-856b-b67af4c294c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:18.348468 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77daab0b-bb6b-4f6d-856b-b67af4c294c9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:18.348750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77daab0b-bb6b-4f6d-856b-b67af4c294c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:18.349028 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77daab0b-bb6b-4f6d-856b-b67af4c294c9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:18.359828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77daab0b-bb6b-4f6d-856b-b67af4c294c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:18.367557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-77daab0b-bb6b-4f6d-856b-b67af4c294c9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 12:20:18.367557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:18.367557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:18.367557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:18.367557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:18.367557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:18.367557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:18.367557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:18.367557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:18.367557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:18.367557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:18.367557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:18.367557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:18.367557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:18.367557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:18.367557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:18.368427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:18.368427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:18.368427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:18.368427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:18.368427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:18.368427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:18.368427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:18.368427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:18.368427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:18.368427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:18.368427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:18.368427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:18.368427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:18.368427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:18.368427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:18.368427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:18.370879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:18.370879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:18.370879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:18.370879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:18.370879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:18.370879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:18.370879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:18.370879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:18.370879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:18.370879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:18.370879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 12:20:18.370879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:18.370879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 135/275] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:18.370879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77daab0b-bb6b-4f6d-856b-b67af4c294c9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:18.390786 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 141/276] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:18.395322 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453f7ca-3ddc-47dd-9a28-bcf8e0d1f577 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:18.486004 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77daab0b-bb6b-4f6d-856b-b67af4c294c9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:18.508321 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453f7ca-3ddc-47dd-9a28-bcf8e0d1f577 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:18.522311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce7a9a-c1d4-4d7f-9727-7f9403a93ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:18.523311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce7a9a-c1d4-4d7f-9727-7f9403a93ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:18.523970 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce7a9a-c1d4-4d7f-9727-7f9403a93ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:18.524274 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce7a9a-c1d4-4d7f-9727-7f9403a93ce0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:18.525457 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce7a9a-c1d4-4d7f-9727-7f9403a93ce0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:18.527146 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce7a9a-c1d4-4d7f-9727-7f9403a93ce0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:18.528935 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 136/277] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:18.529619 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce7a9a-c1d4-4d7f-9727-7f9403a93ce0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:18.547691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f83ae8-4941-47d4-92e7-9ab0eeeb8b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:18.548370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f83ae8-4941-47d4-92e7-9ab0eeeb8b36 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:18.548669 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f83ae8-4941-47d4-92e7-9ab0eeeb8b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:18.548925 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f83ae8-4941-47d4-92e7-9ab0eeeb8b36 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:18.549747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f83ae8-4941-47d4-92e7-9ab0eeeb8b36 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:18.550061 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f83ae8-4941-47d4-92e7-9ab0eeeb8b36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:18.551513 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f83ae8-4941-47d4-92e7-9ab0eeeb8b36 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:18.553074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:18.561516 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 142/278] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 12:20:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:18.562291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f83ae8-4941-47d4-92e7-9ab0eeeb8b36 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:18.588811 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:18.646875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce7a9a-c1d4-4d7f-9727-7f9403a93ce0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:18.667957 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f83ae8-4941-47d4-92e7-9ab0eeeb8b36 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:18.690291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785381b5-f18f-4db0-8efa-5eea94d31559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:18.691218 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785381b5-f18f-4db0-8efa-5eea94d31559 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:18.691489 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785381b5-f18f-4db0-8efa-5eea94d31559 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:18.691705 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785381b5-f18f-4db0-8efa-5eea94d31559 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:18.702665 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785381b5-f18f-4db0-8efa-5eea94d31559 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:18.706052 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6075d55-02a3-4811-90c7-1da0d93c810c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:18.706901 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6075d55-02a3-4811-90c7-1da0d93c810c None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:18.707317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6075d55-02a3-4811-90c7-1da0d93c810c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:18.707715 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6075d55-02a3-4811-90c7-1da0d93c810c None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:18.708542 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6075d55-02a3-4811-90c7-1da0d93c810c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:18.710105 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6075d55-02a3-4811-90c7-1da0d93c810c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:18.710407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-785381b5-f18f-4db0-8efa-5eea94d31559 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:20:18.710407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:18.710407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:18.710407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:18.710407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:18.710407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:18.710407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:18.710407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:18.710407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:18.710407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:18.710407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:18.710407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:18.710407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:18.710407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:18.710407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:18.710407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:18.711287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:18.711287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:18.711287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:18.711287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:18.711287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:18.711287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:18.711287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:18.711287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:18.711287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:18.711287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:18.711287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:18.711287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:18.711287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:18.711287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:18.711287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:18.711287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:18.717479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:18.717479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:18.717479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:18.717479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:18.717479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:18.717479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:18.717479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:18.717479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:18.717479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:18.717479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:18.717479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:20:18.717479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:18.717479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 137/279] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:18.717479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785381b5-f18f-4db0-8efa-5eea94d31559 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:18.750511 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 143/280] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 12:20:18 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:20:18.750511 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6075d55-02a3-4811-90c7-1da0d93c810c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:18.829948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:18.833923 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785381b5-f18f-4db0-8efa-5eea94d31559 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:18.847221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:18.877250 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d4d8c-52ef-4a09-a28a-88e6a7a79e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:18.878606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d4d8c-52ef-4a09-a28a-88e6a7a79e62 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:18.879096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d4d8c-52ef-4a09-a28a-88e6a7a79e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:18.879615 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d4d8c-52ef-4a09-a28a-88e6a7a79e62 None admin] PATH_INFO: `/v3/users/f73e200499404c46a96fee7624386736` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:18.881079 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d4d8c-52ef-4a09-a28a-88e6a7a79e62 None admin] RBAC: Authorizing `identity:get_user(user_id=f73e200499404c46a96fee7624386736)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:18.882757 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d4d8c-52ef-4a09-a28a-88e6a7a79e62 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:18.884606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 138/281] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:20:18 2022] GET /identity/v3/users/f73e200499404c46a96fee7624386736 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:18.898452 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d4d8c-52ef-4a09-a28a-88e6a7a79e62 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:18.898692 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6075d55-02a3-4811-90c7-1da0d93c810c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:18.939497 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201d83d-960e-45ad-98e8-a5b71a2b7067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:18.940835 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201d83d-960e-45ad-98e8-a5b71a2b7067 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:18.941169 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201d83d-960e-45ad-98e8-a5b71a2b7067 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:18.941452 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201d83d-960e-45ad-98e8-a5b71a2b7067 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:18.942211 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201d83d-960e-45ad-98e8-a5b71a2b7067 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:18.944953 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201d83d-960e-45ad-98e8-a5b71a2b7067 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:18.944953 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201d83d-960e-45ad-98e8-a5b71a2b7067 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:18.958039 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 144/282] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 12:20:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:18.963492 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201d83d-960e-45ad-98e8-a5b71a2b7067 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:19.013206 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d4d8c-52ef-4a09-a28a-88e6a7a79e62 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:19.055136 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2e9b4-011f-4c8d-98e8-47553a8efb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:19.056069 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2e9b4-011f-4c8d-98e8-47553a8efb48 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:19.056352 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2e9b4-011f-4c8d-98e8-47553a8efb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:19.056628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba2e9b4-011f-4c8d-98e8-47553a8efb48 None admin] PATH_INFO: `/v3/projects/9cd49b1ddb8042ec8620180f9db712a0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:19.058371 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba2e9b4-011f-4c8d-98e8-47553a8efb48 None admin] RBAC: Authorizing `identity:get_project(project_id=9cd49b1ddb8042ec8620180f9db712a0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:19.061837 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba2e9b4-011f-4c8d-98e8-47553a8efb48 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:19.061837 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 139/283] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:18 2022] GET /identity/v3/projects/9cd49b1ddb8042ec8620180f9db712a0 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:19.067395 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2e9b4-011f-4c8d-98e8-47553a8efb48 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:19.100906 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201d83d-960e-45ad-98e8-a5b71a2b7067 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:19.116399 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:19.134710 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:19.142845 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea1927b-e520-4e13-897c-9139f8f0c0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:19.143741 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1927b-e520-4e13-897c-9139f8f0c0fc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:19.144454 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1927b-e520-4e13-897c-9139f8f0c0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:19.144751 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1927b-e520-4e13-897c-9139f8f0c0fc None admin] PATH_INFO: `/v3/users/91f4188014554c3a82ab08d9fde8fa77` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:19.162110 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea1927b-e520-4e13-897c-9139f8f0c0fc None admin] RBAC: Authorizing `identity:get_user(user_id=91f4188014554c3a82ab08d9fde8fa77)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:19.163929 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea1927b-e520-4e13-897c-9139f8f0c0fc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:19.166161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 145/284] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:20:18 2022] GET /identity/v3/users/91f4188014554c3a82ab08d9fde8fa77 => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:19.172766 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea1927b-e520-4e13-897c-9139f8f0c0fc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:19.201943 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba2e9b4-011f-4c8d-98e8-47553a8efb48 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:19.244019 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ded948-bb97-4020-9124-43a75bf0d1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:19.245074 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ded948-bb97-4020-9124-43a75bf0d1fb None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:19.246062 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ded948-bb97-4020-9124-43a75bf0d1fb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:19.246888 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ded948-bb97-4020-9124-43a75bf0d1fb None admin] PATH_INFO: `/v3/projects/9cd49b1ddb8042ec8620180f9db712a0/users/f73e200499404c46a96fee7624386736/roles/c2e0da29cd7947288c717f7b4b262ef8` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:19.248934 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ded948-bb97-4020-9124-43a75bf0d1fb None admin] RBAC: Authorizing `identity:create_grant(project_id=9cd49b1ddb8042ec8620180f9db712a0, user_id=f73e200499404c46a96fee7624386736, role_id=c2e0da29cd7947288c717f7b4b262ef8)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:19.250398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ded948-bb97-4020-9124-43a75bf0d1fb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:19.271782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 140/285] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 12:20:19 2022] PUT /identity/v3/projects/9cd49b1ddb8042ec8620180f9db712a0/users/f73e200499404c46a96fee7624386736/roles/c2e0da29cd7947288c717f7b4b262ef8 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:20:19.273124 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b16e0-f809-4c73-a9f5-c764c68e6301 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:19.273396 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b16e0-f809-4c73-a9f5-c764c68e6301 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:19.273679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886b16e0-f809-4c73-a9f5-c764c68e6301 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:19.275301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 141/286] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:19.281666 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc11f8-fd06-4b7e-b771-b880fc07b717 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:19.281866 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc11f8-fd06-4b7e-b771-b880fc07b717 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:19.282299 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc11f8-fd06-4b7e-b771-b880fc07b717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:19.307142 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-74dc11f8-fd06-4b7e-b771-b880fc07b717 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:19.308678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea1927b-e520-4e13-897c-9139f8f0c0fc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:19.309675 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-74dc11f8-fd06-4b7e-b771-b880fc07b717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:19.377000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a4ca41-76fe-4a34-900a-f0e82adb7270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:19.377882 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4ca41-76fe-4a34-900a-f0e82adb7270 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:19.378172 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4ca41-76fe-4a34-900a-f0e82adb7270 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:19.378463 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4ca41-76fe-4a34-900a-f0e82adb7270 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:19.379327 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a4ca41-76fe-4a34-900a-f0e82adb7270 None admin] RBAC: Adding query filter params (role.id=3bf034b395b44d88b3af67da01a6ed82, user.id=91f4188014554c3a82ab08d9fde8fa77, scope.system=all) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:19.379636 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a4ca41-76fe-4a34-900a-f0e82adb7270 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:19.382477 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a4ca41-76fe-4a34-900a-f0e82adb7270 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:19.398820 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 146/287] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Jun 29 12:20:19 2022] GET /identity/v3/role_assignments?user.id=91f4188014554c3a82ab08d9fde8fa77&scope.system=all&role.id=3bf034b395b44d88b3af67da01a6ed82 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:19.470551 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 142/288] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:20:19.500954 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a4ca41-76fe-4a34-900a-f0e82adb7270 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:19.547834 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:19.624296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a4ca41-76fe-4a34-900a-f0e82adb7270 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:19.660693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bb1d4f-3baa-44e0-a335-7fb5c2b7642e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:19.661502 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb1d4f-3baa-44e0-a335-7fb5c2b7642e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:19.661785 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb1d4f-3baa-44e0-a335-7fb5c2b7642e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:19.662063 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb1d4f-3baa-44e0-a335-7fb5c2b7642e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:19.673514 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bb1d4f-3baa-44e0-a335-7fb5c2b7642e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:19.681555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f9bb1d4f-3baa-44e0-a335-7fb5c2b7642e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:20:19.681555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:19.681555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:19.681555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:19.681555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:19.681555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:19.681555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:19.681555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:19.681555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:19.681555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:19.681555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:19.681555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:19.681555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:19.681555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:19.681555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:19.681555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:19.682444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:19.682444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:19.682444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:19.682444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:19.682444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:19.682444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:19.682444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:19.682444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:19.682444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:19.682444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:19.682444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:19.682444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:19.682444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:19.682444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:19.682444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:19.682444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:19.683601 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:19.683601 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:19.683601 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:19.683601 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:19.683601 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:19.683601 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:19.683601 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:19.683601 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:19.683601 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:19.683601 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:19.683601 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:20:19.683601 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:19.684210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 147/289] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:19.695577 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc11f8-fd06-4b7e-b771-b880fc07b717 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:19.752632 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:19.840521 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc11f8-fd06-4b7e-b771-b880fc07b717 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:19.895766 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdabe490-4839-4e6b-9778-d6b7f7b4a0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:19.897136 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabe490-4839-4e6b-9778-d6b7f7b4a0c6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:19.897544 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabe490-4839-4e6b-9778-d6b7f7b4a0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:19.897833 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabe490-4839-4e6b-9778-d6b7f7b4a0c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:19.898718 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdabe490-4839-4e6b-9778-d6b7f7b4a0c6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:19.899296 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdabe490-4839-4e6b-9778-d6b7f7b4a0c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:19.900735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdabe490-4839-4e6b-9778-d6b7f7b4a0c6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:19.913183 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 143/290] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 12:20:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:19.919500 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bb1d4f-3baa-44e0-a335-7fb5c2b7642e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:20.080841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:20.087880 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bb1d4f-3baa-44e0-a335-7fb5c2b7642e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:20.137842 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be788dea-cad6-490c-8d78-38ab8bbf6dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:20.138756 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be788dea-cad6-490c-8d78-38ab8bbf6dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:20.139047 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be788dea-cad6-490c-8d78-38ab8bbf6dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:20.139326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be788dea-cad6-490c-8d78-38ab8bbf6dd7 None admin] PATH_INFO: `/v3/users/5d6e0a8cea12467fbb9d6953e1dde05f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:20.140550 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be788dea-cad6-490c-8d78-38ab8bbf6dd7 None admin] RBAC: Authorizing `identity:get_user(user_id=5d6e0a8cea12467fbb9d6953e1dde05f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:20.142003 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be788dea-cad6-490c-8d78-38ab8bbf6dd7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:20.143847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 148/291] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:20:19 2022] GET /identity/v3/users/5d6e0a8cea12467fbb9d6953e1dde05f => generated 297 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:20.149464 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdabe490-4839-4e6b-9778-d6b7f7b4a0c6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:20.274814 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:20.282994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdabe490-4839-4e6b-9778-d6b7f7b4a0c6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:20.333844 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d465f51-72bd-473d-92c1-193863d1a431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:20.334530 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d465f51-72bd-473d-92c1-193863d1a431 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:20.334827 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d465f51-72bd-473d-92c1-193863d1a431 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:20.335112 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d465f51-72bd-473d-92c1-193863d1a431 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:20.426047 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d465f51-72bd-473d-92c1-193863d1a431 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:20.427505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d465f51-72bd-473d-92c1-193863d1a431 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:20.546208 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:20.557526 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 144/292] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:20:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:20:20.563956 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be788dea-cad6-490c-8d78-38ab8bbf6dd7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:20.696541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be788dea-cad6-490c-8d78-38ab8bbf6dd7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:20.749876 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e17be-5d47-4579-8192-6bf282b126ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:20.750808 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e17be-5d47-4579-8192-6bf282b126ac None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:20.751095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e17be-5d47-4579-8192-6bf282b126ac None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:20.751372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e17be-5d47-4579-8192-6bf282b126ac None admin] PATH_INFO: `/v3/projects/9cd49b1ddb8042ec8620180f9db712a0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:20.752676 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2e17be-5d47-4579-8192-6bf282b126ac None admin] RBAC: Authorizing `identity:get_project(project_id=9cd49b1ddb8042ec8620180f9db712a0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:20.754153 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2e17be-5d47-4579-8192-6bf282b126ac None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:20.755966 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 149/293] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:20 2022] GET /identity/v3/projects/9cd49b1ddb8042ec8620180f9db712a0 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:20.761887 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d465f51-72bd-473d-92c1-193863d1a431 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:20.896945 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d465f51-72bd-473d-92c1-193863d1a431 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:20.937853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af74dc2-7a79-4565-aa6a-c830dee946ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:20.938799 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af74dc2-7a79-4565-aa6a-c830dee946ba None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:20.939233 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af74dc2-7a79-4565-aa6a-c830dee946ba None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:20.939660 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af74dc2-7a79-4565-aa6a-c830dee946ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:20.940640 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af74dc2-7a79-4565-aa6a-c830dee946ba None admin] RBAC: Adding query filter params (role.id=3bf034b395b44d88b3af67da01a6ed82, scope.project.id=9cd49b1ddb8042ec8620180f9db712a0, user.id=5d6e0a8cea12467fbb9d6953e1dde05f) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:20.941120 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af74dc2-7a79-4565-aa6a-c830dee946ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:20.942724 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af74dc2-7a79-4565-aa6a-c830dee946ba None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:20.948969 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 145/294] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:20:20 2022] GET /identity/v3/role_assignments?user.id=5d6e0a8cea12467fbb9d6953e1dde05f&scope.project.id=9cd49b1ddb8042ec8620180f9db712a0&role.id=3bf034b395b44d88b3af67da01a6ed82 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:21.669996 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f52f19-3adc-4e28-9a94-dcb4a711f9b9 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:21.670518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f52f19-3adc-4e28-9a94-dcb4a711f9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:21.670518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f52f19-3adc-4e28-9a94-dcb4a711f9b9 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:21.672113 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 150/295] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:21.674962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e2339-cb57-4161-aa74-b0afb010f3ae None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:21.675252 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e2339-cb57-4161-aa74-b0afb010f3ae None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:21.675536 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e2339-cb57-4161-aa74-b0afb010f3ae None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:21.676890 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 146/296] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:21.681100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017579e-ac97-436a-ad6a-94726a426086 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:21.681395 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017579e-ac97-436a-ad6a-94726a426086 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:21.681679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3017579e-ac97-436a-ad6a-94726a426086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:21.684880 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3503dd-47b1-4e8a-ab32-c322a8f8bdaf None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:21.685165 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3503dd-47b1-4e8a-ab32-c322a8f8bdaf None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:21.685443 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3503dd-47b1-4e8a-ab32-c322a8f8bdaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:21.705348 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8d3503dd-47b1-4e8a-ab32-c322a8f8bdaf None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:21.706787 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8d3503dd-47b1-4e8a-ab32-c322a8f8bdaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:21.709274 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3017579e-ac97-436a-ad6a-94726a426086 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:21.711093 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3017579e-ac97-436a-ad6a-94726a426086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:21.740453 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:21.775019 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:21.785127 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 151/297] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:21.786480 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 147/298] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:21.786615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464a415-a03d-4ad8-bf40-88a518634f33 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:21.786803 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464a415-a03d-4ad8-bf40-88a518634f33 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:21.787037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2464a415-a03d-4ad8-bf40-88a518634f33 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:21.787630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103c697-7287-44e3-a009-cd5ffd16139d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:21.787918 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103c697-7287-44e3-a009-cd5ffd16139d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:21.788195 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3103c697-7287-44e3-a009-cd5ffd16139d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:21.788546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 152/299] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:21.793581 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 148/300] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:21.794060 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9754f-b888-4cc1-b26f-075bc378ff1e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:21.794351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9754f-b888-4cc1-b26f-075bc378ff1e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:21.794659 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9754f-b888-4cc1-b26f-075bc378ff1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:21.797822 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afe84d-476a-4492-9d87-2e9e65151efb None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:21.798116 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afe84d-476a-4492-9d87-2e9e65151efb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:21.798404 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afe84d-476a-4492-9d87-2e9e65151efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:21.812063 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1ea9754f-b888-4cc1-b26f-075bc378ff1e None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:21.813250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1ea9754f-b888-4cc1-b26f-075bc378ff1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:21.815162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e4afe84d-476a-4492-9d87-2e9e65151efb None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:21.816322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e4afe84d-476a-4492-9d87-2e9e65151efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:21.880942 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 153/301] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:20:21.881786 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9754f-b888-4cc1-b26f-075bc378ff1e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:21.890724 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 149/302] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:21.890724 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afe84d-476a-4492-9d87-2e9e65151efb None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:21.977900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:21.987041 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9754f-b888-4cc1-b26f-075bc378ff1e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:21.989429 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afe84d-476a-4492-9d87-2e9e65151efb None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:22.009181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:22.021258 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82adfaa5-9d1c-494b-9790-ad9419794cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:22.022057 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adfaa5-9d1c-494b-9790-ad9419794cab None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:22.022347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adfaa5-9d1c-494b-9790-ad9419794cab None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:22.022656 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82adfaa5-9d1c-494b-9790-ad9419794cab None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:22.023958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2295ef-3a5b-4190-b5fb-19f77c4fa555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:22.024751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2295ef-3a5b-4190-b5fb-19f77c4fa555 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:22.025040 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2295ef-3a5b-4190-b5fb-19f77c4fa555 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:22.025319 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2295ef-3a5b-4190-b5fb-19f77c4fa555 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:22.033714 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82adfaa5-9d1c-494b-9790-ad9419794cab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:22.036166 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2295ef-3a5b-4190-b5fb-19f77c4fa555 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:22.041476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-82adfaa5-9d1c-494b-9790-ad9419794cab None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 12:20:22.041476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:22.041476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:22.041476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:22.041476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:22.041476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:22.041476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:22.041476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:22.041476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:22.041476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:22.041476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:22.041476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:22.041476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:22.041476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:22.041476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:22.041476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:22.042747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:22.042747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:22.042747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:22.042747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:22.042747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:22.042747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:22.042747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:22.042747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:22.042747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:22.042747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:22.042747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:22.042747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:22.042747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:22.042747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:22.042747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:22.042747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:22.043623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:22.043623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:22.043623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:22.043623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:22.043623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:22.043623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:22.043623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:22.043623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:22.043623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:22.043623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:22.043623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 12:20:22.043623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:22.043623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 154/303] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:22.044291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82adfaa5-9d1c-494b-9790-ad9419794cab None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:22.044356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6f2295ef-3a5b-4190-b5fb-19f77c4fa555 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 12:20:22.044356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:22.044356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:22.044356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:22.044356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:22.044356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:22.044356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:22.044356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:22.044356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:22.044356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:22.044356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:22.044356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:22.044356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:22.044356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:22.044356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:22.044356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:22.045162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:22.045162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:22.045162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:22.045162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:22.045162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:22.045162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:22.045162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:22.045162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:22.045162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:22.045162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:22.045162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:22.045162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:22.045162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:22.045162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:22.045162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:22.045162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:22.045940 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:22.045940 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:22.045940 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:22.045940 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:22.045940 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:22.045940 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:22.045940 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:22.045940 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:22.045940 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:22.045940 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:22.045940 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 12:20:22.045940 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:22.045940 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 150/304] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:22.046637 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2295ef-3a5b-4190-b5fb-19f77c4fa555 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:22.149326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82adfaa5-9d1c-494b-9790-ad9419794cab None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:22.149956 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2295ef-3a5b-4190-b5fb-19f77c4fa555 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:22.182570 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bae344-d1a9-4c4b-8f64-0eed98e4acd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:22.183484 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bae344-d1a9-4c4b-8f64-0eed98e4acd0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:22.183772 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bae344-d1a9-4c4b-8f64-0eed98e4acd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:22.184053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bae344-d1a9-4c4b-8f64-0eed98e4acd0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:22.184329 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060d29c-cb01-4624-b8f7-2bd88d07739d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:22.185119 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060d29c-cb01-4624-b8f7-2bd88d07739d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:22.185403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060d29c-cb01-4624-b8f7-2bd88d07739d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:22.185683 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b060d29c-cb01-4624-b8f7-2bd88d07739d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:22.186873 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060d29c-cb01-4624-b8f7-2bd88d07739d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:22.188325 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b060d29c-cb01-4624-b8f7-2bd88d07739d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:22.189937 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 151/305] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:22.190727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060d29c-cb01-4624-b8f7-2bd88d07739d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:22.194637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bae344-d1a9-4c4b-8f64-0eed98e4acd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:22.202480 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-57bae344-d1a9-4c4b-8f64-0eed98e4acd0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:20:22.202480 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:22.202480 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:22.202480 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:22.202480 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:22.202480 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:22.202480 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:22.202480 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:22.202480 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:22.202480 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:22.202480 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:22.202480 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:22.202480 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:22.202480 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:22.202480 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:22.202480 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:22.203541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:22.203541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:22.203541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:22.203541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:22.203541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:22.203541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:22.203541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:22.203541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:22.203541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:22.203541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:22.203541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:22.203541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:22.203541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:22.203541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:22.203541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:22.203541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:22.204415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:22.204415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:22.204415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:22.204415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:22.204415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:22.204415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:22.204415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:22.204415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:22.204415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:22.204415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:22.204415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:20:22.204415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:22.204415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 155/306] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:22.205061 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bae344-d1a9-4c4b-8f64-0eed98e4acd0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:22.220256 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:22.246875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:22.297808 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b060d29c-cb01-4624-b8f7-2bd88d07739d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:22.306854 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bae344-d1a9-4c4b-8f64-0eed98e4acd0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:22.334559 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c76a5-a03e-420a-ba6f-afa986fe229c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:22.335228 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c76a5-a03e-420a-ba6f-afa986fe229c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:22.335511 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c76a5-a03e-420a-ba6f-afa986fe229c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:22.335791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c76a5-a03e-420a-ba6f-afa986fe229c None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:22.336541 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c76a5-a03e-420a-ba6f-afa986fe229c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:22.336849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c76a5-a03e-420a-ba6f-afa986fe229c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:22.338508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c76a5-a03e-420a-ba6f-afa986fe229c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:22.342599 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8702f41b-dbca-4254-8ee2-91f49e897b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:22.343285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8702f41b-dbca-4254-8ee2-91f49e897b09 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:22.343573 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8702f41b-dbca-4254-8ee2-91f49e897b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:22.343855 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8702f41b-dbca-4254-8ee2-91f49e897b09 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:22.344643 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8702f41b-dbca-4254-8ee2-91f49e897b09 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:22.344956 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8702f41b-dbca-4254-8ee2-91f49e897b09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:22.346324 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8702f41b-dbca-4254-8ee2-91f49e897b09 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:22.350065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 152/307] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 12:20:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:22.350896 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c76a5-a03e-420a-ba6f-afa986fe229c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:22.358897 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 156/308] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 12:20:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:22.360039 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8702f41b-dbca-4254-8ee2-91f49e897b09 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:22.434285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:22.451722 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c76a5-a03e-420a-ba6f-afa986fe229c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:22.459305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8702f41b-dbca-4254-8ee2-91f49e897b09 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:22.462218 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:22.487839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d279e-2c9d-4e59-9b20-54959b8ca234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:22.488502 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d279e-2c9d-4e59-9b20-54959b8ca234 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:22.488788 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d279e-2c9d-4e59-9b20-54959b8ca234 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:22.489068 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d279e-2c9d-4e59-9b20-54959b8ca234 None admin] PATH_INFO: `/v3/groups` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:22.489763 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d279e-2c9d-4e59-9b20-54959b8ca234 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:22.491235 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4d279e-2c9d-4e59-9b20-54959b8ca234 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:22.492584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4383c811-a49b-484b-ba00-a6cbeeb725c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:22.493244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383c811-a49b-484b-ba00-a6cbeeb725c8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:22.493529 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383c811-a49b-484b-ba00-a6cbeeb725c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:22.493814 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4383c811-a49b-484b-ba00-a6cbeeb725c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:22.494584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4383c811-a49b-484b-ba00-a6cbeeb725c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:22.494892 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4383c811-a49b-484b-ba00-a6cbeeb725c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:22.496259 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4383c811-a49b-484b-ba00-a6cbeeb725c8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:22.504555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 157/309] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 12:20:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:22.515711 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4383c811-a49b-484b-ba00-a6cbeeb725c8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:22.515921 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 153/310] 10.222.0.52 () {60 vars in 1134 bytes} [Wed Jun 29 12:20:22 2022] POST /identity/v3/groups => generated 229 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:20:22.515921 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d279e-2c9d-4e59-9b20-54959b8ca234 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:22.625035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4383c811-a49b-484b-ba00-a6cbeeb725c8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:22.632637 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d279e-2c9d-4e59-9b20-54959b8ca234 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:22.663789 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216bd16b-3c6a-40ed-abf2-3e3dfb69810d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:22.664670 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216bd16b-3c6a-40ed-abf2-3e3dfb69810d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:22.664958 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216bd16b-3c6a-40ed-abf2-3e3dfb69810d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:22.665233 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216bd16b-3c6a-40ed-abf2-3e3dfb69810d None admin] PATH_INFO: `/v3/users/ef2954647e224c3b9579371380a58d2c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:22.670029 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:22.677860 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c16a9-a09c-4826-a4c5-afa5c63ae72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:22.678974 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c16a9-a09c-4826-a4c5-afa5c63ae72d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:22.678974 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c16a9-a09c-4826-a4c5-afa5c63ae72d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:22.679227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c16a9-a09c-4826-a4c5-afa5c63ae72d None admin] PATH_INFO: `/v3/users/f73e200499404c46a96fee7624386736` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:22.680458 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c16a9-a09c-4826-a4c5-afa5c63ae72d None admin] RBAC: Authorizing `identity:get_user(user_id=f73e200499404c46a96fee7624386736)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:22.681812 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c16a9-a09c-4826-a4c5-afa5c63ae72d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:22.684291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216bd16b-3c6a-40ed-abf2-3e3dfb69810d None admin] RBAC: Authorizing `identity:get_user(user_id=ef2954647e224c3b9579371380a58d2c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:22.684408 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 154/311] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:20:22 2022] GET /identity/v3/users/f73e200499404c46a96fee7624386736 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:22.684859 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c16a9-a09c-4826-a4c5-afa5c63ae72d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:22.685577 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216bd16b-3c6a-40ed-abf2-3e3dfb69810d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:22.687327 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 158/312] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:20:22 2022] GET /identity/v3/users/ef2954647e224c3b9579371380a58d2c => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:22.690163 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216bd16b-3c6a-40ed-abf2-3e3dfb69810d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:22.735847 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:22.799843 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216bd16b-3c6a-40ed-abf2-3e3dfb69810d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:22.806237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c16a9-a09c-4826-a4c5-afa5c63ae72d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:22.836734 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a622da-b020-4c20-b6d7-0d50a771bc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:22.837494 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a622da-b020-4c20-b6d7-0d50a771bc17 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:22.837865 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a622da-b020-4c20-b6d7-0d50a771bc17 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:22.838118 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a622da-b020-4c20-b6d7-0d50a771bc17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:22.841982 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04879300-b2fe-4b8b-9a50-ca116540d46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:22.843021 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04879300-b2fe-4b8b-9a50-ca116540d46a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:22.843308 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04879300-b2fe-4b8b-9a50-ca116540d46a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:22.843590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04879300-b2fe-4b8b-9a50-ca116540d46a None admin] PATH_INFO: `/v3/users/91f4188014554c3a82ab08d9fde8fa77` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:22.844798 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04879300-b2fe-4b8b-9a50-ca116540d46a None admin] RBAC: Authorizing `identity:get_user(user_id=91f4188014554c3a82ab08d9fde8fa77)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:22.846243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04879300-b2fe-4b8b-9a50-ca116540d46a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:22.852405 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 155/313] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:20:22 2022] GET /identity/v3/users/91f4188014554c3a82ab08d9fde8fa77 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:22.852405 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04879300-b2fe-4b8b-9a50-ca116540d46a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:22.912535 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:22.920858 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a622da-b020-4c20-b6d7-0d50a771bc17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:22.922324 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a622da-b020-4c20-b6d7-0d50a771bc17 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:22.961851 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04879300-b2fe-4b8b-9a50-ca116540d46a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:22.970597 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:23.000813 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83550e26-1979-4101-823c-0c92433f6fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:23.001470 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83550e26-1979-4101-823c-0c92433f6fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:23.001755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83550e26-1979-4101-823c-0c92433f6fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:23.002039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83550e26-1979-4101-823c-0c92433f6fa9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:23.002899 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83550e26-1979-4101-823c-0c92433f6fa9 None admin] RBAC: Adding query filter params (role.id=c2e0da29cd7947288c717f7b4b262ef8, user.id=ef2954647e224c3b9579371380a58d2c, scope.system=all) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:23.003210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83550e26-1979-4101-823c-0c92433f6fa9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:23.004643 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83550e26-1979-4101-823c-0c92433f6fa9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:23.018500 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 156/314] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Jun 29 12:20:22 2022] GET /identity/v3/role_assignments?user.id=ef2954647e224c3b9579371380a58d2c&scope.system=all&role.id=c2e0da29cd7947288c717f7b4b262ef8 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:23.018500 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83550e26-1979-4101-823c-0c92433f6fa9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:23.040476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 159/315] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:20:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 12:20:23.047420 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a622da-b020-4c20-b6d7-0d50a771bc17 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:23.148798 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83550e26-1979-4101-823c-0c92433f6fa9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:23.169196 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:23.178074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a622da-b020-4c20-b6d7-0d50a771bc17 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:23.187762 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c1e41f-3193-4795-a79a-e8e47e2d9233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:23.188647 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1e41f-3193-4795-a79a-e8e47e2d9233 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:23.188929 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1e41f-3193-4795-a79a-e8e47e2d9233 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:23.189210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1e41f-3193-4795-a79a-e8e47e2d9233 None admin] PATH_INFO: `/v3/system/users/91f4188014554c3a82ab08d9fde8fa77/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:23.190760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c1e41f-3193-4795-a79a-e8e47e2d9233 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=91f4188014554c3a82ab08d9fde8fa77, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:23.192166 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c1e41f-3193-4795-a79a-e8e47e2d9233 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:23.203325 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 157/316] 10.222.0.52 () {58 vars in 1334 bytes} [Wed Jun 29 12:20:23 2022] PUT /identity/v3/system/users/91f4188014554c3a82ab08d9fde8fa77/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:20:23.220116 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e76a0-41f3-41c8-9ca7-2ae5bebeb377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:23.221124 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e76a0-41f3-41c8-9ca7-2ae5bebeb377 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:23.221535 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e76a0-41f3-41c8-9ca7-2ae5bebeb377 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:23.221884 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e76a0-41f3-41c8-9ca7-2ae5bebeb377 None admin] PATH_INFO: `/v3/projects/9cd49b1ddb8042ec8620180f9db712a0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:23.223296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0e76a0-41f3-41c8-9ca7-2ae5bebeb377 None admin] RBAC: Authorizing `identity:get_project(project_id=9cd49b1ddb8042ec8620180f9db712a0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:23.224860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0e76a0-41f3-41c8-9ca7-2ae5bebeb377 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:23.227258 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 160/317] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:23 2022] GET /identity/v3/projects/9cd49b1ddb8042ec8620180f9db712a0 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:23.234215 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c1e41f-3193-4795-a79a-e8e47e2d9233 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:23.283322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:23.361445 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c1e41f-3193-4795-a79a-e8e47e2d9233 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:23.398945 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f76bd-3d9d-4229-9401-55a85b1d923f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:23.399656 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f76bd-3d9d-4229-9401-55a85b1d923f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:23.399945 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f76bd-3d9d-4229-9401-55a85b1d923f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:23.400231 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f76bd-3d9d-4229-9401-55a85b1d923f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:23.401175 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49f76bd-3d9d-4229-9401-55a85b1d923f None admin] RBAC: Adding query filter params (role.id=c2e0da29cd7947288c717f7b4b262ef8, scope.project.id=9cd49b1ddb8042ec8620180f9db712a0, user.id=f73e200499404c46a96fee7624386736) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:23.401608 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49f76bd-3d9d-4229-9401-55a85b1d923f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:23.403120 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49f76bd-3d9d-4229-9401-55a85b1d923f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:23.409709 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 158/318] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:20:23 2022] GET /identity/v3/role_assignments?user.id=f73e200499404c46a96fee7624386736&scope.project.id=9cd49b1ddb8042ec8620180f9db712a0&role.id=c2e0da29cd7947288c717f7b4b262ef8 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:24.628850 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6f7ad-d9fd-4973-92f2-f956b6138b40 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:24.629337 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6f7ad-d9fd-4973-92f2-f956b6138b40 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:24.629337 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6f7ad-d9fd-4973-92f2-f956b6138b40 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:24.634196 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 161/319] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:24.636434 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fe7c7-d802-4623-ae47-be89878d06ab None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:24.636818 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fe7c7-d802-4623-ae47-be89878d06ab None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:24.636978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fe7c7-d802-4623-ae47-be89878d06ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:24.653668 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-373fe7c7-d802-4623-ae47-be89878d06ab None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:24.654867 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-373fe7c7-d802-4623-ae47-be89878d06ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:24.726455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 159/320] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:24.746784 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6f7ad-d9fd-4973-92f2-f956b6138b40 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:24.849849 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6f7ad-d9fd-4973-92f2-f956b6138b40 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:24.884472 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84bef89-1f70-4699-b832-6e44048f5de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:24.885195 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bef89-1f70-4699-b832-6e44048f5de8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:24.885479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bef89-1f70-4699-b832-6e44048f5de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:24.886313 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84bef89-1f70-4699-b832-6e44048f5de8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:24.887536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84bef89-1f70-4699-b832-6e44048f5de8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:24.888967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84bef89-1f70-4699-b832-6e44048f5de8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:24.890770 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 162/321] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:24.896371 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fe7c7-d802-4623-ae47-be89878d06ab None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:24.924277 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:24.998446 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fe7c7-d802-4623-ae47-be89878d06ab None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:25.033020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6fc9bf-9456-4f77-b1fa-839eee0b20ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:25.033678 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fc9bf-9456-4f77-b1fa-839eee0b20ef None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:25.033959 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fc9bf-9456-4f77-b1fa-839eee0b20ef None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:25.034233 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fc9bf-9456-4f77-b1fa-839eee0b20ef None admin] PATH_INFO: `/v3/groups` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:25.034959 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6fc9bf-9456-4f77-b1fa-839eee0b20ef None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:25.036398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6fc9bf-9456-4f77-b1fa-839eee0b20ef None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:25.054623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 160/322] 10.222.0.52 () {60 vars in 1134 bytes} [Wed Jun 29 12:20:24 2022] POST /identity/v3/groups => generated 226 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:20:25.104479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab3d6a-efa3-4781-af85-0cfd117f402f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:25.104783 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab3d6a-efa3-4781-af85-0cfd117f402f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:25.105209 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab3d6a-efa3-4781-af85-0cfd117f402f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:25.106879 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 163/323] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:25.112537 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32817e-9d97-403c-8169-65dca116738f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:25.112827 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32817e-9d97-403c-8169-65dca116738f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:25.113104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32817e-9d97-403c-8169-65dca116738f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:25.131695 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-dd32817e-9d97-403c-8169-65dca116738f None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:25.132904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dd32817e-9d97-403c-8169-65dca116738f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:25.205720 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 161/324] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:25.228295 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab3d6a-efa3-4781-af85-0cfd117f402f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:25.261503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:25.305241 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad70cb8-a3b8-4b2f-bdc5-117128f5651c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:25.305877 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad70cb8-a3b8-4b2f-bdc5-117128f5651c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:25.306232 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad70cb8-a3b8-4b2f-bdc5-117128f5651c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:25.308446 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 162/325] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:25.313945 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0f666-3eff-4d74-95d8-6ef472dfbc4e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:25.314227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0f666-3eff-4d74-95d8-6ef472dfbc4e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:25.314655 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0f666-3eff-4d74-95d8-6ef472dfbc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:25.335195 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e0c0f666-3eff-4d74-95d8-6ef472dfbc4e None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:25.336399 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e0c0f666-3eff-4d74-95d8-6ef472dfbc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:25.348069 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab3d6a-efa3-4781-af85-0cfd117f402f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:25.355530 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:25.395663 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60273003-e7dc-4fee-a9a2-45543e224984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:25.396847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60273003-e7dc-4fee-a9a2-45543e224984 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:25.397133 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60273003-e7dc-4fee-a9a2-45543e224984 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:25.397415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60273003-e7dc-4fee-a9a2-45543e224984 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:25.410928 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60273003-e7dc-4fee-a9a2-45543e224984 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:25.419680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-60273003-e7dc-4fee-a9a2-45543e224984 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 12:20:25.419680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:25.419680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:25.419680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:25.419680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:25.419680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:25.419680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:25.419680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:25.419680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:25.419680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:25.419680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:25.419680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:25.419680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:25.419680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:25.419680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:25.419680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:25.420589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:25.420589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:25.420589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:25.420589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:25.420589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:25.420589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:25.420589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:25.420589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:25.420589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:25.420589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:25.420589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:25.420589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:25.420589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:25.420589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:25.420589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:25.420589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:25.421362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:25.421362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:25.421362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:25.421362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:25.421362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:25.421362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:25.421362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:25.421362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:25.421362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:25.421362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:25.421362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 12:20:25.421362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:25.421362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 164/326] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:25.427413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60273003-e7dc-4fee-a9a2-45543e224984 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:25.428230 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 163/327] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:25.452378 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0f666-3eff-4d74-95d8-6ef472dfbc4e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:25.536343 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:25.552083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60273003-e7dc-4fee-a9a2-45543e224984 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:25.582841 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0f666-3eff-4d74-95d8-6ef472dfbc4e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:25.599122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1472b969-19fa-47ee-b19a-259196edf1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:25.599801 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472b969-19fa-47ee-b19a-259196edf1a0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:25.600085 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472b969-19fa-47ee-b19a-259196edf1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:25.600364 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1472b969-19fa-47ee-b19a-259196edf1a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:25.601119 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1472b969-19fa-47ee-b19a-259196edf1a0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:25.601427 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1472b969-19fa-47ee-b19a-259196edf1a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:25.603043 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1472b969-19fa-47ee-b19a-259196edf1a0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:25.613600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 165/328] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 12:20:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:25.619146 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1472b969-19fa-47ee-b19a-259196edf1a0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:25.631182 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5eeee-b805-4c23-bba7-4d021fbbd1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:25.632840 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5eeee-b805-4c23-bba7-4d021fbbd1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:25.634029 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5eeee-b805-4c23-bba7-4d021fbbd1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:25.634586 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5eeee-b805-4c23-bba7-4d021fbbd1b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:25.649256 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de5eeee-b805-4c23-bba7-4d021fbbd1b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:25.661192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-8de5eeee-b805-4c23-bba7-4d021fbbd1b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:20:25.661192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:25.661192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:25.661192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:25.661192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:25.661192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:25.661192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:25.661192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:25.661192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:25.661192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:25.661192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:25.661192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:25.661192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:25.661192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:25.661192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:25.661192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:25.662293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:25.662293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:25.662293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:25.662293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:25.662293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:25.662293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:25.662293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:25.662293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:25.662293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:25.662293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:25.662293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:25.662293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:25.662293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:25.662293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:25.662293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:25.662293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:25.663335 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:25.663335 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:25.663335 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:25.663335 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:25.663335 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:25.663335 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:25.663335 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:25.663335 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:25.663335 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:25.663335 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:25.663335 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:20:25.663335 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:25.666635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 164/329] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:20:25.674523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5eeee-b805-4c23-bba7-4d021fbbd1b2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:25.694265 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:25.745624 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1472b969-19fa-47ee-b19a-259196edf1a0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:25.788589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6db8b9-581a-4b02-8a38-f1a7703795b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:25.789540 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6db8b9-581a-4b02-8a38-f1a7703795b4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:25.789650 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6db8b9-581a-4b02-8a38-f1a7703795b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:25.789918 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6db8b9-581a-4b02-8a38-f1a7703795b4 None admin] PATH_INFO: `/v3/users/ef2954647e224c3b9579371380a58d2c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:25.791205 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6db8b9-581a-4b02-8a38-f1a7703795b4 None admin] RBAC: Authorizing `identity:get_user(user_id=ef2954647e224c3b9579371380a58d2c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:25.794962 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6db8b9-581a-4b02-8a38-f1a7703795b4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:25.797108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 166/330] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:20:25 2022] GET /identity/v3/users/ef2954647e224c3b9579371380a58d2c => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:25.802535 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6db8b9-581a-4b02-8a38-f1a7703795b4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:25.820086 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5eeee-b805-4c23-bba7-4d021fbbd1b2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:25.831192 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:25.876376 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568aca7-6da9-4f4d-b0d8-87bfc110b9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:25.877249 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568aca7-6da9-4f4d-b0d8-87bfc110b9f2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:25.877628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568aca7-6da9-4f4d-b0d8-87bfc110b9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:25.877913 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568aca7-6da9-4f4d-b0d8-87bfc110b9f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:25.878708 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568aca7-6da9-4f4d-b0d8-87bfc110b9f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:25.879016 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568aca7-6da9-4f4d-b0d8-87bfc110b9f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:25.881360 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568aca7-6da9-4f4d-b0d8-87bfc110b9f2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:25.903094 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 165/331] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 12:20:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:25.913150 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568aca7-6da9-4f4d-b0d8-87bfc110b9f2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:25.959250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6db8b9-581a-4b02-8a38-f1a7703795b4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:26.006017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4944f2-c9b4-49aa-bab2-6a8d32636b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:26.007016 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4944f2-c9b4-49aa-bab2-6a8d32636b9c None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:26.007305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4944f2-c9b4-49aa-bab2-6a8d32636b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:26.007650 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4944f2-c9b4-49aa-bab2-6a8d32636b9c None admin] PATH_INFO: `/v3/system/users/ef2954647e224c3b9579371380a58d2c/roles/c2e0da29cd7947288c717f7b4b262ef8` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:26.009948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4944f2-c9b4-49aa-bab2-6a8d32636b9c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ef2954647e224c3b9579371380a58d2c, role_id=c2e0da29cd7947288c717f7b4b262ef8)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:26.011542 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4944f2-c9b4-49aa-bab2-6a8d32636b9c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:26.016534 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:26.025662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 167/332] 10.222.0.52 () {58 vars in 1334 bytes} [Wed Jun 29 12:20:25 2022] PUT /identity/v3/system/users/ef2954647e224c3b9579371380a58d2c/roles/c2e0da29cd7947288c717f7b4b262ef8 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:20:26.066310 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568aca7-6da9-4f4d-b0d8-87bfc110b9f2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:26.115890 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c2de7c-e46f-4d8b-a1be-6e90f970da69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:26.116845 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2de7c-e46f-4d8b-a1be-6e90f970da69 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:26.117175 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2de7c-e46f-4d8b-a1be-6e90f970da69 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:26.117452 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c2de7c-e46f-4d8b-a1be-6e90f970da69 None admin] PATH_INFO: `/v3/users/91f4188014554c3a82ab08d9fde8fa77` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:26.118851 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c2de7c-e46f-4d8b-a1be-6e90f970da69 None admin] RBAC: Authorizing `identity:get_user(user_id=91f4188014554c3a82ab08d9fde8fa77)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:26.121005 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c2de7c-e46f-4d8b-a1be-6e90f970da69 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:26.123203 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 166/333] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:20:25 2022] GET /identity/v3/users/91f4188014554c3a82ab08d9fde8fa77 => generated 293 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:26.128968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4944f2-c9b4-49aa-bab2-6a8d32636b9c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:26.236532 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:26.254901 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4944f2-c9b4-49aa-bab2-6a8d32636b9c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:26.295523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a84591-5d08-44ae-a816-8b09d9578715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:26.296195 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a84591-5d08-44ae-a816-8b09d9578715 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:26.296479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a84591-5d08-44ae-a816-8b09d9578715 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:26.296757 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a84591-5d08-44ae-a816-8b09d9578715 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:26.297595 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a84591-5d08-44ae-a816-8b09d9578715 None admin] RBAC: Adding query filter params (role.id=3bf034b395b44d88b3af67da01a6ed82, user.id=91f4188014554c3a82ab08d9fde8fa77, scope.system=all) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:26.297908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a84591-5d08-44ae-a816-8b09d9578715 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:26.299382 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a84591-5d08-44ae-a816-8b09d9578715 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:26.312502 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 168/334] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Jun 29 12:20:26 2022] GET /identity/v3/role_assignments?user.id=91f4188014554c3a82ab08d9fde8fa77&scope.system=all&role.id=3bf034b395b44d88b3af67da01a6ed82 => generated 510 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:27.506260 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68bfdc-2cdb-4547-b0d8-56c9d8760fa6 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:27.506804 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68bfdc-2cdb-4547-b0d8-56c9d8760fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:27.506909 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68bfdc-2cdb-4547-b0d8-56c9d8760fa6 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:27.508451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 167/335] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:27.514363 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4eec30-483a-40b0-94ec-51dc812dd1d4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:27.514804 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4eec30-483a-40b0-94ec-51dc812dd1d4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:27.514995 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4eec30-483a-40b0-94ec-51dc812dd1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:27.539414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-aa4eec30-483a-40b0-94ec-51dc812dd1d4 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:27.540624 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-aa4eec30-483a-40b0-94ec-51dc812dd1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:27.615954 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 169/336] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:27.640381 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68bfdc-2cdb-4547-b0d8-56c9d8760fa6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:27.763747 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68bfdc-2cdb-4547-b0d8-56c9d8760fa6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:27.808051 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d5e7e-b17d-44ff-82b5-cd223101eac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:27.808854 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d5e7e-b17d-44ff-82b5-cd223101eac8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:27.809128 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d5e7e-b17d-44ff-82b5-cd223101eac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:27.809403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d5e7e-b17d-44ff-82b5-cd223101eac8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:27.818465 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:27.822736 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9d5e7e-b17d-44ff-82b5-cd223101eac8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:27.833828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-3e9d5e7e-b17d-44ff-82b5-cd223101eac8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:20:27.833828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:27.833828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:27.833828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:27.833828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:27.833828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:27.833828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:27.833828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:27.833828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:27.833828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:27.833828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:27.833828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:27.833828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:27.833828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:27.833828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:27.833828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:27.834776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:27.834776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:27.834776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:27.834776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:27.834776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:27.834776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:27.834776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:27.834776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:27.834776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:27.834776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:27.834776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:27.834776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:27.834776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:27.834776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:27.834776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:27.834776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:27.835569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:27.835569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:27.835569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:27.835569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:27.835569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:27.835569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:27.835569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:27.835569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:27.835569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:27.835569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:27.835569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:20:27.835569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:27.835569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 168/337] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:27.841484 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4eec30-483a-40b0-94ec-51dc812dd1d4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:27.907852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe79789-98a4-4068-8409-a9406edc7154 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:27.908142 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe79789-98a4-4068-8409-a9406edc7154 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:27.908321 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe79789-98a4-4068-8409-a9406edc7154 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:27.909688 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 169/338] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:20:27.915451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f69192-c72f-4c94-a336-a0708b66822d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:27.915747 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f69192-c72f-4c94-a336-a0708b66822d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:27.916019 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f69192-c72f-4c94-a336-a0708b66822d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:27.943526 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:27.948998 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-55f69192-c72f-4c94-a336-a0708b66822d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:27.950776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-55f69192-c72f-4c94-a336-a0708b66822d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:28.013146 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4eec30-483a-40b0-94ec-51dc812dd1d4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:28.072350 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 170/339] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:28.092108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281b769-6563-4d88-858e-fd875531234b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:28.092799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281b769-6563-4d88-858e-fd875531234b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:28.093085 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281b769-6563-4d88-858e-fd875531234b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:28.093362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281b769-6563-4d88-858e-fd875531234b None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:28.094125 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281b769-6563-4d88-858e-fd875531234b None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:28.094455 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281b769-6563-4d88-858e-fd875531234b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:28.095876 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281b769-6563-4d88-858e-fd875531234b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:28.112314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f69192-c72f-4c94-a336-a0708b66822d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:28.113787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 170/340] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 12:20:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:28.119132 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281b769-6563-4d88-858e-fd875531234b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:28.303283 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f69192-c72f-4c94-a336-a0708b66822d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:28.353554 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:28.369077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af6719-a10d-476b-a62c-c185b4930d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:28.369077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af6719-a10d-476b-a62c-c185b4930d77 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:28.369077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af6719-a10d-476b-a62c-c185b4930d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:28.369077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af6719-a10d-476b-a62c-c185b4930d77 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:28.372650 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281b769-6563-4d88-858e-fd875531234b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:28.383519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af6719-a10d-476b-a62c-c185b4930d77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:28.402147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a2af6719-a10d-476b-a62c-c185b4930d77 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 12:20:28.402147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:28.402147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:28.402147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:28.402147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:28.402147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:28.402147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:28.402147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:28.402147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:28.402147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:28.402147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:28.402147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:28.402147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:28.402147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:28.402147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:28.402147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:28.403227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:28.403227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:28.403227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:28.403227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:28.403227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:28.403227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:28.403227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:28.403227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:28.403227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:28.403227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:28.403227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:28.403227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:28.403227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:28.403227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:28.403227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:28.403227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:28.404096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:28.404096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:28.404096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:28.404096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:28.404096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:28.404096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:28.404096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:28.404096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:28.404096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:28.404096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:28.404096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 12:20:28.404096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:28.404096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 171/341] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 12:20:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:20:28.409306 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af6719-a10d-476b-a62c-c185b4930d77 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:28.467173 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207053e0-ff8a-49c3-8c07-63d20735fff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:28.467849 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207053e0-ff8a-49c3-8c07-63d20735fff1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:28.468143 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207053e0-ff8a-49c3-8c07-63d20735fff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:28.468428 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207053e0-ff8a-49c3-8c07-63d20735fff1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:28.482815 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:28.561049 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af6719-a10d-476b-a62c-c185b4930d77 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:28.591282 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207053e0-ff8a-49c3-8c07-63d20735fff1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:28.592769 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207053e0-ff8a-49c3-8c07-63d20735fff1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:28.615984 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97175c7-6f0f-40f2-87ca-42a6e1f386a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:28.616662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97175c7-6f0f-40f2-87ca-42a6e1f386a3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:28.616952 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97175c7-6f0f-40f2-87ca-42a6e1f386a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:28.617236 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97175c7-6f0f-40f2-87ca-42a6e1f386a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:28.617995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97175c7-6f0f-40f2-87ca-42a6e1f386a3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:28.618305 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97175c7-6f0f-40f2-87ca-42a6e1f386a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:28.619868 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97175c7-6f0f-40f2-87ca-42a6e1f386a3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:28.634307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 172/342] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:20:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:28.640106 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97175c7-6f0f-40f2-87ca-42a6e1f386a3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:28.727925 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:28.772393 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97175c7-6f0f-40f2-87ca-42a6e1f386a3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:28.793316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 171/343] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:20:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 674 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 12:20:28.798652 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207053e0-ff8a-49c3-8c07-63d20735fff1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:28.813110 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f8098-d925-4d9b-b2aa-f8ce3b560f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:28.813849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f8098-d925-4d9b-b2aa-f8ce3b560f55 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:28.814236 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f8098-d925-4d9b-b2aa-f8ce3b560f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:28.814646 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f8098-d925-4d9b-b2aa-f8ce3b560f55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:28.842247 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:28.894904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f8098-d925-4d9b-b2aa-f8ce3b560f55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:28.896815 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f8098-d925-4d9b-b2aa-f8ce3b560f55 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:28.943656 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207053e0-ff8a-49c3-8c07-63d20735fff1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:28.974812 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:28.984661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941e678-2503-4eac-9702-ebff9f7dc0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:28.985539 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941e678-2503-4eac-9702-ebff9f7dc0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:28.985825 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941e678-2503-4eac-9702-ebff9f7dc0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:28.986100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941e678-2503-4eac-9702-ebff9f7dc0d5 None admin] PATH_INFO: `/v3/projects/a19c893c70c84649bb5406d2cf9e2708` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:28.987363 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e941e678-2503-4eac-9702-ebff9f7dc0d5 None admin] RBAC: Authorizing `identity:get_project(project_id=a19c893c70c84649bb5406d2cf9e2708)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:28.988820 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e941e678-2503-4eac-9702-ebff9f7dc0d5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:28.990555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 172/344] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:28 2022] GET /identity/v3/projects/a19c893c70c84649bb5406d2cf9e2708 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:28.996763 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941e678-2503-4eac-9702-ebff9f7dc0d5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:29.013202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 173/345] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:20:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 12:20:29.019919 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f8098-d925-4d9b-b2aa-f8ce3b560f55 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:29.121846 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941e678-2503-4eac-9702-ebff9f7dc0d5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:29.124322 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:29.147847 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f8098-d925-4d9b-b2aa-f8ce3b560f55 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:29.185678 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97001d36-696a-46af-b034-2481e6ea5634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:29.186583 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97001d36-696a-46af-b034-2481e6ea5634 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:29.186875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97001d36-696a-46af-b034-2481e6ea5634 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:29.187157 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97001d36-696a-46af-b034-2481e6ea5634 None admin] PATH_INFO: `/v3/projects/a19c893c70c84649bb5406d2cf9e2708` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:29.187264 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b983f-e923-4c97-9d8c-6c0d51d8723f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:29.188033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b983f-e923-4c97-9d8c-6c0d51d8723f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:29.188323 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b983f-e923-4c97-9d8c-6c0d51d8723f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:29.188426 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97001d36-696a-46af-b034-2481e6ea5634 None admin] RBAC: Authorizing `identity:get_project(project_id=a19c893c70c84649bb5406d2cf9e2708)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:29.188594 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b983f-e923-4c97-9d8c-6c0d51d8723f None admin] PATH_INFO: `/v3/groups/cca510b0461648fe9c9ef63cb8839147` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:29.189807 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97001d36-696a-46af-b034-2481e6ea5634 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:29.191986 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 174/346] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:29 2022] GET /identity/v3/projects/a19c893c70c84649bb5406d2cf9e2708 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:29.195363 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b983f-e923-4c97-9d8c-6c0d51d8723f None admin] RBAC: Authorizing `identity:get_group(group_id=cca510b0461648fe9c9ef63cb8839147)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:29.196832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b983f-e923-4c97-9d8c-6c0d51d8723f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:29.197351 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97001d36-696a-46af-b034-2481e6ea5634 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:29.199355 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 173/347] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 12:20:28 2022] GET /identity/v3/groups/cca510b0461648fe9c9ef63cb8839147 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:29.204984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b983f-e923-4c97-9d8c-6c0d51d8723f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:29.226817 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:29.319651 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97001d36-696a-46af-b034-2481e6ea5634 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:29.326408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b983f-e923-4c97-9d8c-6c0d51d8723f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:29.366377 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad150a76-dee9-4991-b8e5-8ce900cb6c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:29.367286 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad150a76-dee9-4991-b8e5-8ce900cb6c41 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:29.367568 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad150a76-dee9-4991-b8e5-8ce900cb6c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:29.368032 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad150a76-dee9-4991-b8e5-8ce900cb6c41 None admin] PATH_INFO: `/v3/groups/cca510b0461648fe9c9ef63cb8839147` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:29.369827 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad150a76-dee9-4991-b8e5-8ce900cb6c41 None admin] RBAC: Authorizing `identity:get_group(group_id=cca510b0461648fe9c9ef63cb8839147)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:29.372116 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad150a76-dee9-4991-b8e5-8ce900cb6c41 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:29.374591 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 175/348] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 12:20:29 2022] GET /identity/v3/groups/cca510b0461648fe9c9ef63cb8839147 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:29.380468 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad150a76-dee9-4991-b8e5-8ce900cb6c41 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:29.403740 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a4cda-269b-4647-b8f1-ad418211b9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:29.404774 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a4cda-269b-4647-b8f1-ad418211b9fb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:29.405257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a4cda-269b-4647-b8f1-ad418211b9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:29.405745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a4cda-269b-4647-b8f1-ad418211b9fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:29.406832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a4cda-269b-4647-b8f1-ad418211b9fb None admin] RBAC: Adding query filter params (group.id=cca510b0461648fe9c9ef63cb8839147, role.id=3bf034b395b44d88b3af67da01a6ed82, scope.project.id=a19c893c70c84649bb5406d2cf9e2708) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:29.407449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a4cda-269b-4647-b8f1-ad418211b9fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:29.409123 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a4cda-269b-4647-b8f1-ad418211b9fb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:29.422663 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 174/349] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jun 29 12:20:29 2022] GET /identity/v3/role_assignments?group.id=cca510b0461648fe9c9ef63cb8839147&scope.project.id=a19c893c70c84649bb5406d2cf9e2708&role.id=3bf034b395b44d88b3af67da01a6ed82 => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:29.484587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:29.486039 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d079b-e031-4670-864c-9759ce1a4e00 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:29.486786 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d079b-e031-4670-864c-9759ce1a4e00 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:29.487280 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d079b-e031-4670-864c-9759ce1a4e00 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:29.489181 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 175/350] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:29.493574 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad150a76-dee9-4991-b8e5-8ce900cb6c41 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:29.495854 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe45c80-8514-4c53-a9bd-c5f4d43b74fc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:29.496357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe45c80-8514-4c53-a9bd-c5f4d43b74fc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:29.496848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe45c80-8514-4c53-a9bd-c5f4d43b74fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:29.518153 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-efe45c80-8514-4c53-a9bd-c5f4d43b74fc None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:29.519791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-efe45c80-8514-4c53-a9bd-c5f4d43b74fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:29.531395 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c422e-5a4a-4cb5-bc3f-f87a9e1b9150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:29.532131 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c422e-5a4a-4cb5-bc3f-f87a9e1b9150 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:29.532426 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c422e-5a4a-4cb5-bc3f-f87a9e1b9150 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:29.532762 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c422e-5a4a-4cb5-bc3f-f87a9e1b9150 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:29.533621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c422e-5a4a-4cb5-bc3f-f87a9e1b9150 None admin] RBAC: Adding query filter params (group.id=cca510b0461648fe9c9ef63cb8839147, role.id=6005bd32383a4706a69108f7a2cf3ed1, scope.project.id=a19c893c70c84649bb5406d2cf9e2708) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:29.533939 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c422e-5a4a-4cb5-bc3f-f87a9e1b9150 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:29.535615 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c422e-5a4a-4cb5-bc3f-f87a9e1b9150 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:29.542607 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 176/351] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jun 29 12:20:29 2022] GET /identity/v3/role_assignments?group.id=cca510b0461648fe9c9ef63cb8839147&scope.project.id=a19c893c70c84649bb5406d2cf9e2708&role.id=6005bd32383a4706a69108f7a2cf3ed1 => generated 260 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:29.550989 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:29.602412 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 176/352] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:20:29.639031 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c422e-5a4a-4cb5-bc3f-f87a9e1b9150 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:29.696921 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8267a09-5ed2-4cce-b81b-7be58e1d03fc None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:29.697555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8267a09-5ed2-4cce-b81b-7be58e1d03fc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:29.698064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8267a09-5ed2-4cce-b81b-7be58e1d03fc None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:29.700104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 177/353] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:29.705864 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d25de-4604-4f77-aab2-c684c759d712 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:29.706490 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d25de-4604-4f77-aab2-c684c759d712 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:29.706975 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d25de-4604-4f77-aab2-c684c759d712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:29.726276 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dd2d25de-4604-4f77-aab2-c684c759d712 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:29.727859 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dd2d25de-4604-4f77-aab2-c684c759d712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:29.801886 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 178/354] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:20:29.827249 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d25de-4604-4f77-aab2-c684c759d712 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:29.832591 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c422e-5a4a-4cb5-bc3f-f87a9e1b9150 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:29.918595 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c6bae-eec6-4c43-9d70-67dcb455d7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:29.919331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c6bae-eec6-4c43-9d70-67dcb455d7ca None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:29.919611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c6bae-eec6-4c43-9d70-67dcb455d7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:29.919887 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c6bae-eec6-4c43-9d70-67dcb455d7ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:29.933429 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:29.939964 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d25de-4604-4f77-aab2-c684c759d712 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:29.955075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c6bae-eec6-4c43-9d70-67dcb455d7ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:29.963035 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-745c6bae-eec6-4c43-9d70-67dcb455d7ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:20:29.963035 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:29.963035 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:29.963035 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:29.963035 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:29.963035 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:29.963035 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:29.963035 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:29.963035 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:29.963035 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:29.963035 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:29.963035 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:29.963035 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:29.963035 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:29.963035 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:29.963035 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:29.964622 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:29.964622 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:29.964622 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:29.964622 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:29.964622 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:29.964622 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:29.964622 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:29.964622 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:29.964622 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:29.964622 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:29.964622 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:29.964622 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:29.964622 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:29.964622 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:29.964622 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:29.964622 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:29.965448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:29.965448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:29.965448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:29.965448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:29.965448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:29.965448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:29.965448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:29.965448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:29.965448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:29.965448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:29.965448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:20:29.965448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:29.965448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 177/355] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:29.966506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626a620-215f-4445-98cb-9034d4f500e2 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:29.966967 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626a620-215f-4445-98cb-9034d4f500e2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:29.967398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626a620-215f-4445-98cb-9034d4f500e2 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:29.968824 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 178/356] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:29.974460 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edddd4d-4037-401e-9d1f-19348adfc5b1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:29.974760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edddd4d-4037-401e-9d1f-19348adfc5b1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:29.975044 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edddd4d-4037-401e-9d1f-19348adfc5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:29.985522 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903bbf2a-5bd3-402c-9bad-97e28517073d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:29.986741 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903bbf2a-5bd3-402c-9bad-97e28517073d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:29.987435 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903bbf2a-5bd3-402c-9bad-97e28517073d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:29.987979 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903bbf2a-5bd3-402c-9bad-97e28517073d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:29.998798 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7edddd4d-4037-401e-9d1f-19348adfc5b1 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:30.000246 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7edddd4d-4037-401e-9d1f-19348adfc5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:30.001982 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903bbf2a-5bd3-402c-9bad-97e28517073d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:30.010312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-903bbf2a-5bd3-402c-9bad-97e28517073d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 12:20:30.010312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:30.010312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:30.010312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:30.010312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:30.010312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:30.010312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:30.010312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:30.010312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:30.010312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:30.010312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:30.010312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:30.010312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:30.010312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:30.010312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:30.010312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:30.011817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:30.011817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:30.011817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:30.011817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:30.011817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:30.011817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:30.011817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:30.011817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:30.011817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:30.011817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:30.011817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:30.011817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:30.011817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:30.011817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:30.011817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:30.011817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:30.012610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:30.012610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:30.012610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:30.012610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:30.012610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:30.012610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:30.012610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:30.012610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:30.012610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:30.012610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:30.012610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 29 12:20:30.012610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:30.015322 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 179/357] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:20:30.016336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903bbf2a-5bd3-402c-9bad-97e28517073d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:30.018313 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:30.084469 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 179/358] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:20:30.085401 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edddd4d-4037-401e-9d1f-19348adfc5b1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:30.141906 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903bbf2a-5bd3-402c-9bad-97e28517073d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:30.181252 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747abdd-182e-4622-85db-076295cde2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:30.181971 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747abdd-182e-4622-85db-076295cde2eb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:30.182266 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747abdd-182e-4622-85db-076295cde2eb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:30.183184 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747abdd-182e-4622-85db-076295cde2eb None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:30.183953 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747abdd-182e-4622-85db-076295cde2eb None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:30.184316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747abdd-182e-4622-85db-076295cde2eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:30.185697 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747abdd-182e-4622-85db-076295cde2eb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:30.194988 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 180/359] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 12:20:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:30.195700 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747abdd-182e-4622-85db-076295cde2eb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:30.222805 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:30.312623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747abdd-182e-4622-85db-076295cde2eb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:30.350413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d73b16-594d-4d16-9dbd-b2cb7285ac2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:30.351257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d73b16-594d-4d16-9dbd-b2cb7285ac2a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:30.351547 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d73b16-594d-4d16-9dbd-b2cb7285ac2a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:30.351827 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d73b16-594d-4d16-9dbd-b2cb7285ac2a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:30.366357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d73b16-594d-4d16-9dbd-b2cb7285ac2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:30.376001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-64d73b16-594d-4d16-9dbd-b2cb7285ac2a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 12:20:30.376001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:30.376001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:30.376001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:30.376001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:30.376001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:30.376001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:30.376001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:30.376001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:30.376001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:30.376001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:30.376001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:30.376001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:30.376001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:30.376001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:30.376001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:30.376992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:30.376992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:30.376992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:30.376992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:30.376992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:30.376992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:30.376992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:30.376992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:30.376992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:30.376992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:30.376992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:30.376992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:30.376992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:30.376992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:30.376992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:30.376992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:30.377799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:30.377799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:30.377799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:30.377799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:30.377799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:30.377799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:30.377799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:30.377799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:30.377799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:30.377799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:30.377799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 12:20:30.377799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:30.377799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 181/360] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 12:20:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:30.378486 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d73b16-594d-4d16-9dbd-b2cb7285ac2a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:30.397724 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edddd4d-4037-401e-9d1f-19348adfc5b1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:30.468216 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e38e49-a8d8-4de9-a37d-726b79973ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:30.468855 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e38e49-a8d8-4de9-a37d-726b79973ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:30.469120 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e38e49-a8d8-4de9-a37d-726b79973ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:30.469403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e38e49-a8d8-4de9-a37d-726b79973ba7 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:30.470166 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e38e49-a8d8-4de9-a37d-726b79973ba7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:30.470502 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e38e49-a8d8-4de9-a37d-726b79973ba7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:30.471891 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e38e49-a8d8-4de9-a37d-726b79973ba7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:30.483002 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 180/361] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 12:20:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:30.484228 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e38e49-a8d8-4de9-a37d-726b79973ba7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:30.484853 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:30.496874 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:30.530612 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d73b16-594d-4d16-9dbd-b2cb7285ac2a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:30.578042 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cfd6c4-5d26-426a-8ac0-bd15085de6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:30.579026 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cfd6c4-5d26-426a-8ac0-bd15085de6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:30.579525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cfd6c4-5d26-426a-8ac0-bd15085de6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:30.580022 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cfd6c4-5d26-426a-8ac0-bd15085de6b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:30.626042 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e38e49-a8d8-4de9-a37d-726b79973ba7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:30.659987 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cfd6c4-5d26-426a-8ac0-bd15085de6b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:30.662128 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cfd6c4-5d26-426a-8ac0-bd15085de6b0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:30.673658 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e1a64-73ae-4c04-a75b-336d60db2b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:30.674341 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e1a64-73ae-4c04-a75b-336d60db2b1e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:30.674666 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e1a64-73ae-4c04-a75b-336d60db2b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:30.674954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e1a64-73ae-4c04-a75b-336d60db2b1e None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:30.676086 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e1a64-73ae-4c04-a75b-336d60db2b1e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:30.676409 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e1a64-73ae-4c04-a75b-336d60db2b1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:30.678144 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e1a64-73ae-4c04-a75b-336d60db2b1e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:30.690307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 181/362] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:20:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:30.692115 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41f2ed-976b-4d96-a2c7-d54ec8676a60 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:30.692569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41f2ed-976b-4d96-a2c7-d54ec8676a60 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:30.693174 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41f2ed-976b-4d96-a2c7-d54ec8676a60 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:30.695583 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 182/363] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:30.696440 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41f2ed-976b-4d96-a2c7-d54ec8676a60 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:30.770008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:30.804152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 182/364] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:20:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:20:30.805151 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cfd6c4-5d26-426a-8ac0-bd15085de6b0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:30.834811 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:30.860391 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41f2ed-976b-4d96-a2c7-d54ec8676a60 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:30.903180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7796dd-bec1-4388-9c48-794e40065f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:30.904492 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7796dd-bec1-4388-9c48-794e40065f27 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:30.905523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7796dd-bec1-4388-9c48-794e40065f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:30.905969 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7796dd-bec1-4388-9c48-794e40065f27 None admin] PATH_INFO: `/v3/users/ef2954647e224c3b9579371380a58d2c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:30.907591 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7796dd-bec1-4388-9c48-794e40065f27 None admin] RBAC: Authorizing `identity:get_user(user_id=ef2954647e224c3b9579371380a58d2c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:30.909347 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7796dd-bec1-4388-9c48-794e40065f27 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:30.911682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 183/365] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:20:30 2022] GET /identity/v3/users/ef2954647e224c3b9579371380a58d2c => generated 293 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:30.913072 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58a9cf-4785-481c-ac94-53111aaefaa5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:30.913488 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58a9cf-4785-481c-ac94-53111aaefaa5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:30.913914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58a9cf-4785-481c-ac94-53111aaefaa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:30.932123 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-fc58a9cf-4785-481c-ac94-53111aaefaa5 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:30.933521 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cfd6c4-5d26-426a-8ac0-bd15085de6b0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:30.934340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fc58a9cf-4785-481c-ac94-53111aaefaa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:30.974316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226a86f-4cbf-4ba5-9261-05284dd9c0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:30.975375 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226a86f-4cbf-4ba5-9261-05284dd9c0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:30.975858 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226a86f-4cbf-4ba5-9261-05284dd9c0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:30.976353 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e226a86f-4cbf-4ba5-9261-05284dd9c0f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:31.013075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 184/366] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:31.013075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58a9cf-4785-481c-ac94-53111aaefaa5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:31.027976 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:31.072287 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e226a86f-4cbf-4ba5-9261-05284dd9c0f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:31.074796 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e226a86f-4cbf-4ba5-9261-05284dd9c0f7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:31.081895 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:31.124783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58a9cf-4785-481c-ac94-53111aaefaa5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:31.181855 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f2acb-b20d-454d-a929-55277e41e942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:31.183375 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f2acb-b20d-454d-a929-55277e41e942 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:31.183663 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f2acb-b20d-454d-a929-55277e41e942 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:31.184052 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f2acb-b20d-454d-a929-55277e41e942 None admin] PATH_INFO: `/v3/projects/9cd49b1ddb8042ec8620180f9db712a0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:31.185273 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f2acb-b20d-454d-a929-55277e41e942 None admin] RBAC: Authorizing `identity:get_project(project_id=9cd49b1ddb8042ec8620180f9db712a0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:31.187311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6f2acb-b20d-454d-a929-55277e41e942 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:31.190285 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 185/367] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:31 2022] GET /identity/v3/projects/9cd49b1ddb8042ec8620180f9db712a0 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:31.191961 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f2acb-b20d-454d-a929-55277e41e942 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:31.208523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 183/368] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:20:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:20:31.209759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226a86f-4cbf-4ba5-9261-05284dd9c0f7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:31.286799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:31.319553 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e226a86f-4cbf-4ba5-9261-05284dd9c0f7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:31.321949 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6f2acb-b20d-454d-a929-55277e41e942 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:31.357954 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0d381-97f4-4781-bbfa-3ed6fee9d240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:31.359211 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0d381-97f4-4781-bbfa-3ed6fee9d240 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:31.359698 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0d381-97f4-4781-bbfa-3ed6fee9d240 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:31.360183 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0d381-97f4-4781-bbfa-3ed6fee9d240 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:31.363488 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:31.372680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df848eed-8f4f-4f2d-bbf2-35713db4bea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:31.373709 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df848eed-8f4f-4f2d-bbf2-35713db4bea2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:31.374164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df848eed-8f4f-4f2d-bbf2-35713db4bea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:31.374634 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df848eed-8f4f-4f2d-bbf2-35713db4bea2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:31.375490 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df848eed-8f4f-4f2d-bbf2-35713db4bea2 None admin] RBAC: Adding query filter params (role.id=c2e0da29cd7947288c717f7b4b262ef8, user.id=ef2954647e224c3b9579371380a58d2c, scope.system=all) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:31.375961 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df848eed-8f4f-4f2d-bbf2-35713db4bea2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:31.378065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df848eed-8f4f-4f2d-bbf2-35713db4bea2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:31.378798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a0d381-97f4-4781-bbfa-3ed6fee9d240 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:31.387622 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-37a0d381-97f4-4781-bbfa-3ed6fee9d240 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:20:31.387622 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:31.387622 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:31.387622 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:31.387622 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:31.387622 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:31.387622 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:31.387622 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:31.387622 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:31.387622 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:31.387622 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:31.387622 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:31.387622 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:31.387622 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:31.387622 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:31.387622 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:31.388537 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:31.388537 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:31.388537 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:31.388537 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:31.388537 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:31.388537 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:31.388537 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:31.388537 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:31.388537 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:31.388537 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:31.388537 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:31.388537 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:31.388537 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:31.388537 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:31.388537 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:31.388537 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:31.389347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:31.389347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:31.389347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:31.389347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:31.389347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:31.389347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:31.389347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:31.389347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:31.389347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:31.389347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:31.389347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:20:31.389347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:31.391793 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 184/369] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:20:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:31.392835 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0d381-97f4-4781-bbfa-3ed6fee9d240 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:31.395529 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 186/370] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Jun 29 12:20:31 2022] GET /identity/v3/role_assignments?user.id=ef2954647e224c3b9579371380a58d2c&scope.system=all&role.id=c2e0da29cd7947288c717f7b4b262ef8 => generated 510 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:31.397271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df848eed-8f4f-4f2d-bbf2-35713db4bea2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:31.508594 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df848eed-8f4f-4f2d-bbf2-35713db4bea2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:31.548318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532a5ac-30e9-4ad5-b37a-c2a3a4668697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:31.548318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532a5ac-30e9-4ad5-b37a-c2a3a4668697 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:31.548318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532a5ac-30e9-4ad5-b37a-c2a3a4668697 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:31.548318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532a5ac-30e9-4ad5-b37a-c2a3a4668697 None admin] PATH_INFO: `/v3/projects/9cd49b1ddb8042ec8620180f9db712a0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:31.549591 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f532a5ac-30e9-4ad5-b37a-c2a3a4668697 None admin] RBAC: Authorizing `identity:get_project(project_id=9cd49b1ddb8042ec8620180f9db712a0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:31.551131 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f532a5ac-30e9-4ad5-b37a-c2a3a4668697 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:31.553108 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 187/371] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:31 2022] GET /identity/v3/projects/9cd49b1ddb8042ec8620180f9db712a0 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:31.553741 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532a5ac-30e9-4ad5-b37a-c2a3a4668697 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:31.558450 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0d381-97f4-4781-bbfa-3ed6fee9d240 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:31.596562 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:31.604879 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf4252-a342-4fb4-ae58-bb4d17580f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:31.606039 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf4252-a342-4fb4-ae58-bb4d17580f02 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:31.606589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf4252-a342-4fb4-ae58-bb4d17580f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:31.607065 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf4252-a342-4fb4-ae58-bb4d17580f02 None admin] PATH_INFO: `/v3/groups/cca510b0461648fe9c9ef63cb8839147` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:31.608477 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf4252-a342-4fb4-ae58-bb4d17580f02 None admin] RBAC: Authorizing `identity:get_group(group_id=cca510b0461648fe9c9ef63cb8839147)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:31.610174 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf4252-a342-4fb4-ae58-bb4d17580f02 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:31.612492 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 185/372] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 12:20:31 2022] GET /identity/v3/groups/cca510b0461648fe9c9ef63cb8839147 => generated 226 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:31.613631 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf4252-a342-4fb4-ae58-bb4d17580f02 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:31.623454 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:31.696422 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532a5ac-30e9-4ad5-b37a-c2a3a4668697 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:31.732775 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf4252-a342-4fb4-ae58-bb4d17580f02 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:31.754001 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca7411-684f-4293-94fe-b9293c5b4786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:31.756889 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca7411-684f-4293-94fe-b9293c5b4786 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:31.759139 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca7411-684f-4293-94fe-b9293c5b4786 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:31.759685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca7411-684f-4293-94fe-b9293c5b4786 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:31.761276 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ca7411-684f-4293-94fe-b9293c5b4786 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:31.763211 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ca7411-684f-4293-94fe-b9293c5b4786 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:31.764877 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ca7411-684f-4293-94fe-b9293c5b4786 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:31.784597 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 188/373] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:31.788097 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca7411-684f-4293-94fe-b9293c5b4786 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:31.806176 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e272a6-c597-4838-9405-7536e0348bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:31.807942 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e272a6-c597-4838-9405-7536e0348bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:31.808254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e272a6-c597-4838-9405-7536e0348bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:31.808543 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e272a6-c597-4838-9405-7536e0348bd0 None admin] PATH_INFO: `/v3/groups/cca510b0461648fe9c9ef63cb8839147` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:31.810065 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e272a6-c597-4838-9405-7536e0348bd0 None admin] RBAC: Authorizing `identity:get_group(group_id=cca510b0461648fe9c9ef63cb8839147)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:31.811886 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e272a6-c597-4838-9405-7536e0348bd0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:31.814598 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 186/374] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 12:20:31 2022] GET /identity/v3/groups/cca510b0461648fe9c9ef63cb8839147 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:31.815679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e272a6-c597-4838-9405-7536e0348bd0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:31.906671 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:31.919141 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca7411-684f-4293-94fe-b9293c5b4786 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:31.940589 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:31.956734 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e272a6-c597-4838-9405-7536e0348bd0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:31.961408 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc681836-47d6-4c7d-8d36-036f0801878e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:31.962302 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc681836-47d6-4c7d-8d36-036f0801878e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:31.962792 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc681836-47d6-4c7d-8d36-036f0801878e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:31.963245 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc681836-47d6-4c7d-8d36-036f0801878e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:31.964509 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc681836-47d6-4c7d-8d36-036f0801878e None admin] RBAC: Adding query filter params (group.id=cca510b0461648fe9c9ef63cb8839147, role.id=3bf034b395b44d88b3af67da01a6ed82, scope.project.id=9cd49b1ddb8042ec8620180f9db712a0) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:31.965061 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc681836-47d6-4c7d-8d36-036f0801878e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:31.966833 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc681836-47d6-4c7d-8d36-036f0801878e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:31.973719 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 189/375] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jun 29 12:20:31 2022] GET /identity/v3/role_assignments?group.id=cca510b0461648fe9c9ef63cb8839147&scope.project.id=9cd49b1ddb8042ec8620180f9db712a0&role.id=3bf034b395b44d88b3af67da01a6ed82 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:31.974744 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc681836-47d6-4c7d-8d36-036f0801878e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:31.997382 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d96ff-665e-4c7d-b30e-57c75176df5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:31.998546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d96ff-665e-4c7d-b30e-57c75176df5f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:31.999041 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d96ff-665e-4c7d-b30e-57c75176df5f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:31.999521 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d96ff-665e-4c7d-b30e-57c75176df5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:32.098119 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d96ff-665e-4c7d-b30e-57c75176df5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:32.100274 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d96ff-665e-4c7d-b30e-57c75176df5f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:32.180420 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc681836-47d6-4c7d-8d36-036f0801878e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:32.216457 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:32.230962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9caceb8-0684-454c-8adf-555298d14ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:32.231635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9caceb8-0684-454c-8adf-555298d14ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:32.231920 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9caceb8-0684-454c-8adf-555298d14ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:32.232199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9caceb8-0684-454c-8adf-555298d14ff9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:32.233156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9caceb8-0684-454c-8adf-555298d14ff9 None admin] RBAC: Adding query filter params (group.id=cca510b0461648fe9c9ef63cb8839147, role.id=6005bd32383a4706a69108f7a2cf3ed1, scope.project.id=9cd49b1ddb8042ec8620180f9db712a0) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:32.233480 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9caceb8-0684-454c-8adf-555298d14ff9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:32.234985 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9caceb8-0684-454c-8adf-555298d14ff9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:32.241375 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 190/376] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jun 29 12:20:31 2022] GET /identity/v3/role_assignments?group.id=cca510b0461648fe9c9ef63cb8839147&scope.project.id=9cd49b1ddb8042ec8620180f9db712a0&role.id=6005bd32383a4706a69108f7a2cf3ed1 => generated 260 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:32.242665 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0931e04-d11a-4bf2-b39d-1954ad360a63 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:32.242953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0931e04-d11a-4bf2-b39d-1954ad360a63 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:32.243240 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0931e04-d11a-4bf2-b39d-1954ad360a63 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:32.244961 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 191/377] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:32.251324 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5f5a0-8778-478e-b8c6-fe5daf3f4049 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:32.251923 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5f5a0-8778-478e-b8c6-fe5daf3f4049 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:32.252182 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5f5a0-8778-478e-b8c6-fe5daf3f4049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:32.271960 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 187/378] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:20:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:20:32.274872 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-42a5f5a0-8778-478e-b8c6-fe5daf3f4049 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:32.276279 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-42a5f5a0-8778-478e-b8c6-fe5daf3f4049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:32.278845 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d96ff-665e-4c7d-b30e-57c75176df5f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:32.330348 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:32.378865 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 192/379] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:32.383827 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618fe2f-e75b-46ca-9e5c-2f1e6bc8cf5a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:32.384122 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618fe2f-e75b-46ca-9e5c-2f1e6bc8cf5a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:32.384407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7618fe2f-e75b-46ca-9e5c-2f1e6bc8cf5a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:32.387101 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 193/380] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:20:32.402410 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12790ad2-b2cb-4c74-8c19-2d3203477f84 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:32.402881 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12790ad2-b2cb-4c74-8c19-2d3203477f84 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:32.403265 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12790ad2-b2cb-4c74-8c19-2d3203477f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:32.422084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-12790ad2-b2cb-4c74-8c19-2d3203477f84 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:32.423331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-12790ad2-b2cb-4c74-8c19-2d3203477f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:32.461919 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d96ff-665e-4c7d-b30e-57c75176df5f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:32.504825 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2fa0c7-b0b2-42db-a8ed-2080fe420aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:32.505767 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fa0c7-b0b2-42db-a8ed-2080fe420aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:32.506053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fa0c7-b0b2-42db-a8ed-2080fe420aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:32.506380 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fa0c7-b0b2-42db-a8ed-2080fe420aa0 None admin] PATH_INFO: `/v3/projects/a19795d69941402bbd95f8c9172fb339` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:32.507833 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2fa0c7-b0b2-42db-a8ed-2080fe420aa0 None admin] RBAC: Authorizing `identity:get_project(project_id=a19795d69941402bbd95f8c9172fb339)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:32.509049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2fa0c7-b0b2-42db-a8ed-2080fe420aa0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:32.512213 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 188/381] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:32 2022] GET /identity/v3/projects/a19795d69941402bbd95f8c9172fb339 => generated 344 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:32.513295 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2fa0c7-b0b2-42db-a8ed-2080fe420aa0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:32.515177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 194/382] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:20:32.521364 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12790ad2-b2cb-4c74-8c19-2d3203477f84 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:32.561564 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:32.643678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2fa0c7-b0b2-42db-a8ed-2080fe420aa0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:32.673437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12790ad2-b2cb-4c74-8c19-2d3203477f84 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:32.683894 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:32.689025 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf7100-0412-41d6-bf4c-93dd6699f55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:32.690368 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf7100-0412-41d6-bf4c-93dd6699f55e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:32.690818 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf7100-0412-41d6-bf4c-93dd6699f55e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:32.691450 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf7100-0412-41d6-bf4c-93dd6699f55e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:32.711186 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf7100-0412-41d6-bf4c-93dd6699f55e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:32.718137 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14321fae-a031-409a-a0b6-6e1886f5895a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:32.719037 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14321fae-a031-409a-a0b6-6e1886f5895a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:32.719325 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14321fae-a031-409a-a0b6-6e1886f5895a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:32.719609 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14321fae-a031-409a-a0b6-6e1886f5895a None admin] PATH_INFO: `/v3/groups/21405215d71d47b9b7884dde246ff5e1` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:32.722215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9ccf7100-0412-41d6-bf4c-93dd6699f55e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:20:32.722215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:32.722215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:32.722215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:32.722215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:32.722215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:32.722215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:32.722215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:32.722215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:32.722215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:32.722215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:32.722215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:32.722215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:32.722215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:32.722215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:32.722215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:32.723276 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:32.723276 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:32.723276 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:32.723276 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:32.723276 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:32.723276 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:32.723276 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:32.723276 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:32.723276 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:32.723276 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:32.723276 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:32.723276 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:32.723276 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:32.723276 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:32.723276 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:32.723276 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:32.724263 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:32.724263 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:32.724263 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:32.724263 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:32.724263 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:32.724263 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:32.724263 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:32.724263 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:32.724263 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:32.724263 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:32.724263 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:20:32.724263 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:32.727781 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 189/383] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:32.731515 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf7100-0412-41d6-bf4c-93dd6699f55e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:32.732886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14321fae-a031-409a-a0b6-6e1886f5895a None admin] RBAC: Authorizing `identity:get_group(group_id=21405215d71d47b9b7884dde246ff5e1)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:32.734273 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14321fae-a031-409a-a0b6-6e1886f5895a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:32.737238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 195/384] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 12:20:32 2022] GET /identity/v3/groups/21405215d71d47b9b7884dde246ff5e1 => generated 229 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:32.738038 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14321fae-a031-409a-a0b6-6e1886f5895a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:32.849810 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:32.860818 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf7100-0412-41d6-bf4c-93dd6699f55e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:32.867139 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14321fae-a031-409a-a0b6-6e1886f5895a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:32.905278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ef13b-b5ae-445d-b094-f0deb266ecb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:32.905462 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56dcb0-49c2-4bb7-af66-7fe9de2ee087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:32.906159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56dcb0-49c2-4bb7-af66-7fe9de2ee087 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:32.906469 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56dcb0-49c2-4bb7-af66-7fe9de2ee087 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:32.906760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56dcb0-49c2-4bb7-af66-7fe9de2ee087 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:32.907194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ef13b-b5ae-445d-b094-f0deb266ecb4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:32.907517 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc56dcb0-49c2-4bb7-af66-7fe9de2ee087 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:32.907831 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc56dcb0-49c2-4bb7-af66-7fe9de2ee087 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:32.908055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ef13b-b5ae-445d-b094-f0deb266ecb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:32.908522 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ef13b-b5ae-445d-b094-f0deb266ecb4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:32.909275 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc56dcb0-49c2-4bb7-af66-7fe9de2ee087 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:32.919666 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 196/385] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 12:20:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:32.920486 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56dcb0-49c2-4bb7-af66-7fe9de2ee087 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:32.922596 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39ef13b-b5ae-445d-b094-f0deb266ecb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:32.933203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e39ef13b-b5ae-445d-b094-f0deb266ecb4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 12:20:32.933203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:32.933203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:32.933203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:32.933203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:32.933203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:32.933203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:32.933203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:32.933203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:32.933203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:32.933203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:32.933203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:32.933203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:32.933203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:32.933203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:32.933203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:32.934194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:32.934194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:32.934194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:32.934194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:32.934194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:32.934194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:32.934194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:32.934194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:32.934194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:32.934194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:32.934194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:32.934194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:32.934194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:32.934194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:32.934194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:32.934194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:32.935036 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:32.935036 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:32.935036 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:32.935036 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:32.935036 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:32.935036 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:32.935036 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:32.935036 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:32.935036 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:32.935036 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:32.935036 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 12:20:32.935036 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:32.937905 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 190/386] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 12:20:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:32.939111 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ef13b-b5ae-445d-b094-f0deb266ecb4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:32.959642 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:33.051750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56dcb0-49c2-4bb7-af66-7fe9de2ee087 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:33.066306 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ef13b-b5ae-445d-b094-f0deb266ecb4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:33.094579 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f345358-7d8c-41c7-90ce-dc907296f895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:33.095634 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f345358-7d8c-41c7-90ce-dc907296f895 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:33.096063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f345358-7d8c-41c7-90ce-dc907296f895 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:33.096807 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f345358-7d8c-41c7-90ce-dc907296f895 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:33.098118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f345358-7d8c-41c7-90ce-dc907296f895 None admin] RBAC: Adding query filter params (group.id=21405215d71d47b9b7884dde246ff5e1, role.id=3a7f4bf556344407a6d9540c89d13641, scope.project.id=a19795d69941402bbd95f8c9172fb339) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:33.098599 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f345358-7d8c-41c7-90ce-dc907296f895 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:33.100650 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f345358-7d8c-41c7-90ce-dc907296f895 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:33.109050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 197/387] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jun 29 12:20:32 2022] GET /identity/v3/role_assignments?group.id=21405215d71d47b9b7884dde246ff5e1&scope.project.id=a19795d69941402bbd95f8c9172fb339&role.id=3a7f4bf556344407a6d9540c89d13641 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:33.110155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f345358-7d8c-41c7-90ce-dc907296f895 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:33.110400 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8432fa-51d6-4cb9-a7a3-4e8e64b6367f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:33.111541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8432fa-51d6-4cb9-a7a3-4e8e64b6367f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:33.111933 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8432fa-51d6-4cb9-a7a3-4e8e64b6367f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:33.112316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8432fa-51d6-4cb9-a7a3-4e8e64b6367f None admin] PATH_INFO: `/v3/groups/cca510b0461648fe9c9ef63cb8839147` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:33.113800 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8432fa-51d6-4cb9-a7a3-4e8e64b6367f None admin] RBAC: Authorizing `identity:get_group(group_id=cca510b0461648fe9c9ef63cb8839147)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:33.115641 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8432fa-51d6-4cb9-a7a3-4e8e64b6367f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:33.117579 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 191/388] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 12:20:32 2022] GET /identity/v3/groups/cca510b0461648fe9c9ef63cb8839147 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:33.123758 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8432fa-51d6-4cb9-a7a3-4e8e64b6367f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:33.156760 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:33.243397 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:33.252123 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8432fa-51d6-4cb9-a7a3-4e8e64b6367f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:33.253645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f345358-7d8c-41c7-90ce-dc907296f895 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:33.293621 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7cd438-1e50-4c10-a41c-2a18c55525ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:33.294700 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7cd438-1e50-4c10-a41c-2a18c55525ef None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:33.294928 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7cd438-1e50-4c10-a41c-2a18c55525ef None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:33.295216 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7cd438-1e50-4c10-a41c-2a18c55525ef None admin] PATH_INFO: `/v3/projects/a19c893c70c84649bb5406d2cf9e2708` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:33.296546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7cd438-1e50-4c10-a41c-2a18c55525ef None admin] RBAC: Authorizing `identity:get_project(project_id=a19c893c70c84649bb5406d2cf9e2708)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:33.298303 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7cd438-1e50-4c10-a41c-2a18c55525ef None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:33.300765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 192/389] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:33 2022] GET /identity/v3/projects/a19c893c70c84649bb5406d2cf9e2708 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:33.304615 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455483af-119f-43b5-9ab9-29f8aeefb10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:33.305312 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455483af-119f-43b5-9ab9-29f8aeefb10d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:33.305598 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455483af-119f-43b5-9ab9-29f8aeefb10d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:33.305880 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455483af-119f-43b5-9ab9-29f8aeefb10d None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:33.306680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455483af-119f-43b5-9ab9-29f8aeefb10d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:33.306996 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455483af-119f-43b5-9ab9-29f8aeefb10d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:33.309156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455483af-119f-43b5-9ab9-29f8aeefb10d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:33.311469 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7cd438-1e50-4c10-a41c-2a18c55525ef None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:33.333523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 198/390] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:20:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:33.340787 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455483af-119f-43b5-9ab9-29f8aeefb10d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:33.457882 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:33.469512 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7cd438-1e50-4c10-a41c-2a18c55525ef None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:33.489564 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455483af-119f-43b5-9ab9-29f8aeefb10d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:33.505817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58768fb-f4e8-46fa-97ac-28c2d9f150d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:33.506914 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58768fb-f4e8-46fa-97ac-28c2d9f150d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:33.507307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58768fb-f4e8-46fa-97ac-28c2d9f150d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:33.507712 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58768fb-f4e8-46fa-97ac-28c2d9f150d8 None admin] PATH_INFO: `/v3/projects/a19c893c70c84649bb5406d2cf9e2708/groups/cca510b0461648fe9c9ef63cb8839147/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:33.510063 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58768fb-f4e8-46fa-97ac-28c2d9f150d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=a19c893c70c84649bb5406d2cf9e2708, group_id=cca510b0461648fe9c9ef63cb8839147, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:33.512119 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58768fb-f4e8-46fa-97ac-28c2d9f150d8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:33.537457 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 193/391] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Jun 29 12:20:33 2022] PUT /identity/v3/projects/a19c893c70c84649bb5406d2cf9e2708/groups/cca510b0461648fe9c9ef63cb8839147/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:20:33.540532 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa5a73-a099-4273-9d9d-5927d25c2797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:33.541638 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa5a73-a099-4273-9d9d-5927d25c2797 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:33.542196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa5a73-a099-4273-9d9d-5927d25c2797 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:33.542682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa5a73-a099-4273-9d9d-5927d25c2797 None admin] PATH_INFO: `/v3/groups/cca510b0461648fe9c9ef63cb8839147` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:33.545292 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa5a73-a099-4273-9d9d-5927d25c2797 None admin] RBAC: Authorizing `identity:get_group(group_id=cca510b0461648fe9c9ef63cb8839147)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:33.547107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa5a73-a099-4273-9d9d-5927d25c2797 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:33.549454 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 199/392] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 12:20:33 2022] GET /identity/v3/groups/cca510b0461648fe9c9ef63cb8839147 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:33.554708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58768fb-f4e8-46fa-97ac-28c2d9f150d8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:33.677534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58768fb-f4e8-46fa-97ac-28c2d9f150d8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:33.724439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a4d62-19e8-4f8c-9729-b916beea39f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:33.725581 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a4d62-19e8-4f8c-9729-b916beea39f3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:33.726012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a4d62-19e8-4f8c-9729-b916beea39f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:33.726446 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a4d62-19e8-4f8c-9729-b916beea39f3 None admin] PATH_INFO: `/v3/projects/a19c893c70c84649bb5406d2cf9e2708` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:33.727820 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a4d62-19e8-4f8c-9729-b916beea39f3 None admin] RBAC: Authorizing `identity:get_project(project_id=a19c893c70c84649bb5406d2cf9e2708)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:33.729999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a4d62-19e8-4f8c-9729-b916beea39f3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:33.732593 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 194/393] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:33 2022] GET /identity/v3/projects/a19c893c70c84649bb5406d2cf9e2708 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:33.738059 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa5a73-a099-4273-9d9d-5927d25c2797 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:33.773266 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:33.864051 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa5a73-a099-4273-9d9d-5927d25c2797 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:33.903657 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36a90b0-fc90-42d6-9bc6-8bb2772752d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:33.904555 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a90b0-fc90-42d6-9bc6-8bb2772752d9 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:33.904839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a90b0-fc90-42d6-9bc6-8bb2772752d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:33.905118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36a90b0-fc90-42d6-9bc6-8bb2772752d9 None admin] PATH_INFO: `/v3/projects/a19c893c70c84649bb5406d2cf9e2708/groups/cca510b0461648fe9c9ef63cb8839147/roles/6005bd32383a4706a69108f7a2cf3ed1` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:33.910398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36a90b0-fc90-42d6-9bc6-8bb2772752d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=a19c893c70c84649bb5406d2cf9e2708, group_id=cca510b0461648fe9c9ef63cb8839147, role_id=6005bd32383a4706a69108f7a2cf3ed1)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:33.911900 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36a90b0-fc90-42d6-9bc6-8bb2772752d9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:33.931276 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 200/394] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Jun 29 12:20:33 2022] PUT /identity/v3/projects/a19c893c70c84649bb5406d2cf9e2708/groups/cca510b0461648fe9c9ef63cb8839147/roles/6005bd32383a4706a69108f7a2cf3ed1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:20:34.663035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65730091-b916-4022-a56d-bbf6165b2e03 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:34.663963 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65730091-b916-4022-a56d-bbf6165b2e03 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:34.664470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65730091-b916-4022-a56d-bbf6165b2e03 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:34.666286 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 195/395] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:34.684701 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7753c9d-d7f2-4a77-8724-905dcb7bc288 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:34.684997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7753c9d-d7f2-4a77-8724-905dcb7bc288 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:34.685268 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7753c9d-d7f2-4a77-8724-905dcb7bc288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:34.707433 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d7753c9d-d7f2-4a77-8724-905dcb7bc288 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:34.708638 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d7753c9d-d7f2-4a77-8724-905dcb7bc288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:34.794256 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 201/396] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:34.829300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65730091-b916-4022-a56d-bbf6165b2e03 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:34.860035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:34.868928 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46915723-2188-40ce-819d-855084bebe8a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:34.869219 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46915723-2188-40ce-819d-855084bebe8a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:34.869501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46915723-2188-40ce-819d-855084bebe8a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:34.874711 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 202/397] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:34.880416 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1b903-628b-4a22-97d4-e70d9498cef4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:34.880675 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1b903-628b-4a22-97d4-e70d9498cef4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:34.880954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1b903-628b-4a22-97d4-e70d9498cef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:34.911588 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9fd1b903-628b-4a22-97d4-e70d9498cef4 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:34.912907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9fd1b903-628b-4a22-97d4-e70d9498cef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:34.931925 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:34.974650 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65730091-b916-4022-a56d-bbf6165b2e03 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:35.030644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b6f0c-a621-4814-b8fd-37539bfe4e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:35.031457 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b6f0c-a621-4814-b8fd-37539bfe4e34 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:35.031936 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b6f0c-a621-4814-b8fd-37539bfe4e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:35.032372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373b6f0c-a621-4814-b8fd-37539bfe4e34 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:35.044756 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 203/398] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:20:35.046878 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373b6f0c-a621-4814-b8fd-37539bfe4e34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:35.055838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-373b6f0c-a621-4814-b8fd-37539bfe4e34 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:20:35.055838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:35.055838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:35.055838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:35.055838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:35.055838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:35.055838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:35.055838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:35.055838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:35.055838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:35.055838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:35.055838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:35.055838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:35.055838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:35.055838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:35.055838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:35.057064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:35.057064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:35.057064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:35.057064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:35.057064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:35.057064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:35.057064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:35.057064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:35.057064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:35.057064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:35.057064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:35.057064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:35.057064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:35.057064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:35.057064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:35.057064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:35.058870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:35.058870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:35.058870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:35.058870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:35.058870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:35.058870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:35.058870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:35.058870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:35.058870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:35.058870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:35.058870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:20:35.058870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:35.063506 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 196/399] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:35.066316 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1b903-628b-4a22-97d4-e70d9498cef4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:35.079635 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b6f0c-a621-4814-b8fd-37539bfe4e34 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:35.178479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:35.185189 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1b903-628b-4a22-97d4-e70d9498cef4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:35.200908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373b6f0c-a621-4814-b8fd-37539bfe4e34 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:35.224018 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d4570-677e-4c3b-8561-e1a5db47c3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:35.225088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d4570-677e-4c3b-8561-e1a5db47c3af None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:35.225576 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d4570-677e-4c3b-8561-e1a5db47c3af None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:35.226082 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d4570-677e-4c3b-8561-e1a5db47c3af None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:35.228454 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d4570-677e-4c3b-8561-e1a5db47c3af None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:35.228975 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d4570-677e-4c3b-8561-e1a5db47c3af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:35.231543 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d4570-677e-4c3b-8561-e1a5db47c3af None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:35.238045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9d437-cc11-4a2c-8722-4c3ca42dbc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:35.239258 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9d437-cc11-4a2c-8722-4c3ca42dbc2e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:35.239258 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9d437-cc11-4a2c-8722-4c3ca42dbc2e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:35.239567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9d437-cc11-4a2c-8722-4c3ca42dbc2e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:35.244290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 204/400] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 12:20:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:35.250134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d4570-677e-4c3b-8561-e1a5db47c3af None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:35.251901 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b9d437-cc11-4a2c-8722-4c3ca42dbc2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:35.261074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-20b9d437-cc11-4a2c-8722-4c3ca42dbc2e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 12:20:35.261074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:35.261074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:35.261074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:35.261074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:35.261074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:35.261074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:35.261074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:35.261074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:35.261074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:35.261074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:35.261074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:35.261074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:35.261074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:35.261074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:35.261074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:35.261999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:35.261999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:35.261999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:35.261999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:35.261999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:35.261999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:35.261999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:35.261999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:35.261999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:35.261999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:35.261999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:35.261999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:35.261999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:35.261999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:35.261999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:35.261999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:35.264198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:35.264198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:35.264198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:35.264198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:35.264198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:35.264198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:35.264198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:35.264198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:35.264198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:35.264198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:35.264198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 12:20:35.264198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:35.264198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 197/401] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 12:20:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:20:35.267986 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:35.271721 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9d437-cc11-4a2c-8722-4c3ca42dbc2e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:35.379651 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d4570-677e-4c3b-8561-e1a5db47c3af None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:35.424294 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7703b23a-6230-4316-97ff-253abcff7c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:35.425563 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703b23a-6230-4316-97ff-253abcff7c66 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:35.426032 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703b23a-6230-4316-97ff-253abcff7c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:35.426598 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7703b23a-6230-4316-97ff-253abcff7c66 None admin] PATH_INFO: `/v3/groups/cca510b0461648fe9c9ef63cb8839147` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:35.427998 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7703b23a-6230-4316-97ff-253abcff7c66 None admin] RBAC: Authorizing `identity:get_group(group_id=cca510b0461648fe9c9ef63cb8839147)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:35.429728 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7703b23a-6230-4316-97ff-253abcff7c66 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:35.432027 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 205/402] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 12:20:35 2022] GET /identity/v3/groups/cca510b0461648fe9c9ef63cb8839147 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:35.439833 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7703b23a-6230-4316-97ff-253abcff7c66 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:35.486930 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9d437-cc11-4a2c-8722-4c3ca42dbc2e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:35.529795 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02fd70-15cf-43db-9c4a-39d1623fadd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:35.529795 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02fd70-15cf-43db-9c4a-39d1623fadd4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:35.530155 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02fd70-15cf-43db-9c4a-39d1623fadd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:35.530379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02fd70-15cf-43db-9c4a-39d1623fadd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:35.531187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02fd70-15cf-43db-9c4a-39d1623fadd4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:35.531493 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02fd70-15cf-43db-9c4a-39d1623fadd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:35.532887 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02fd70-15cf-43db-9c4a-39d1623fadd4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:35.538682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:35.543652 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 198/403] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:20:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:35.549579 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02fd70-15cf-43db-9c4a-39d1623fadd4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:35.553733 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:35.581797 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7703b23a-6230-4316-97ff-253abcff7c66 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:35.626954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979ae9d-47cd-40ce-b02d-d02d75dbf373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:35.628168 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979ae9d-47cd-40ce-b02d-d02d75dbf373 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:35.628676 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979ae9d-47cd-40ce-b02d-d02d75dbf373 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:35.629180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979ae9d-47cd-40ce-b02d-d02d75dbf373 None admin] PATH_INFO: `/v3/projects/9cd49b1ddb8042ec8620180f9db712a0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:35.634683 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f979ae9d-47cd-40ce-b02d-d02d75dbf373 None admin] RBAC: Authorizing `identity:get_project(project_id=9cd49b1ddb8042ec8620180f9db712a0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:35.636752 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f979ae9d-47cd-40ce-b02d-d02d75dbf373 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:35.639717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 206/404] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:35 2022] GET /identity/v3/projects/9cd49b1ddb8042ec8620180f9db712a0 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:35.644104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979ae9d-47cd-40ce-b02d-d02d75dbf373 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:35.663177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02fd70-15cf-43db-9c4a-39d1623fadd4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:35.702010 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0af059-d9c0-406f-b88b-1868ffe61ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:35.702820 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0af059-d9c0-406f-b88b-1868ffe61ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:35.703104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0af059-d9c0-406f-b88b-1868ffe61ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:35.703380 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0af059-d9c0-406f-b88b-1868ffe61ad2 None admin] PATH_INFO: `/v3/groups/cca510b0461648fe9c9ef63cb8839147` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:35.704572 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0af059-d9c0-406f-b88b-1868ffe61ad2 None admin] RBAC: Authorizing `identity:get_group(group_id=cca510b0461648fe9c9ef63cb8839147)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:35.706122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0af059-d9c0-406f-b88b-1868ffe61ad2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:35.711793 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 199/405] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 12:20:35 2022] GET /identity/v3/groups/cca510b0461648fe9c9ef63cb8839147 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:35.714627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0af059-d9c0-406f-b88b-1868ffe61ad2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:35.830770 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:35.832100 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979ae9d-47cd-40ce-b02d-d02d75dbf373 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:35.876280 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0af059-d9c0-406f-b88b-1868ffe61ad2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:35.881907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792e242-783b-4d71-8fcf-928cb73ecd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:35.883886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792e242-783b-4d71-8fcf-928cb73ecd67 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:35.889203 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792e242-783b-4d71-8fcf-928cb73ecd67 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:35.889762 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9792e242-783b-4d71-8fcf-928cb73ecd67 None admin] PATH_INFO: `/v3/projects/9cd49b1ddb8042ec8620180f9db712a0/groups/cca510b0461648fe9c9ef63cb8839147/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:35.892122 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9792e242-783b-4d71-8fcf-928cb73ecd67 None admin] RBAC: Authorizing `identity:create_grant(project_id=9cd49b1ddb8042ec8620180f9db712a0, group_id=cca510b0461648fe9c9ef63cb8839147, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:35.893795 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9792e242-783b-4d71-8fcf-928cb73ecd67 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:35.913292 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 207/406] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Jun 29 12:20:35 2022] PUT /identity/v3/projects/9cd49b1ddb8042ec8620180f9db712a0/groups/cca510b0461648fe9c9ef63cb8839147/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:20:35.953392 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5674b65a-5bd8-45b8-b412-f8104c87a120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:35.959606 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674b65a-5bd8-45b8-b412-f8104c87a120 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:35.959927 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674b65a-5bd8-45b8-b412-f8104c87a120 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:35.960172 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5674b65a-5bd8-45b8-b412-f8104c87a120 None admin] PATH_INFO: `/v3/projects/9cd49b1ddb8042ec8620180f9db712a0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:35.961575 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674b65a-5bd8-45b8-b412-f8104c87a120 None admin] RBAC: Authorizing `identity:get_project(project_id=9cd49b1ddb8042ec8620180f9db712a0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:35.963062 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5674b65a-5bd8-45b8-b412-f8104c87a120 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:35.965750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 200/407] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:35 2022] GET /identity/v3/projects/9cd49b1ddb8042ec8620180f9db712a0 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:35.971626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792e242-783b-4d71-8fcf-928cb73ecd67 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:35.980832 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:36.063279 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e0972-fb92-495d-81da-8dc1b4706daf None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:36.063618 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e0972-fb92-495d-81da-8dc1b4706daf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:36.063842 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e0972-fb92-495d-81da-8dc1b4706daf None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:36.065236 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 201/408] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:20:36.071505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c75232-ec74-4964-8bf7-36b09de3004d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:36.071826 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c75232-ec74-4964-8bf7-36b09de3004d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:36.072085 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c75232-ec74-4964-8bf7-36b09de3004d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:36.119491 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-56c75232-ec74-4964-8bf7-36b09de3004d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:36.121267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-56c75232-ec74-4964-8bf7-36b09de3004d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:36.147289 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9792e242-783b-4d71-8fcf-928cb73ecd67 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:36.189466 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e6144-3917-437c-9729-b065c287e8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:36.190366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e6144-3917-437c-9729-b065c287e8be None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:36.190681 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e6144-3917-437c-9729-b065c287e8be None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:36.190965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e6144-3917-437c-9729-b065c287e8be None admin] PATH_INFO: `/v3/projects/9cd49b1ddb8042ec8620180f9db712a0/groups/cca510b0461648fe9c9ef63cb8839147/roles/6005bd32383a4706a69108f7a2cf3ed1` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:36.194795 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9e6144-3917-437c-9729-b065c287e8be None admin] RBAC: Authorizing `identity:create_grant(project_id=9cd49b1ddb8042ec8620180f9db712a0, group_id=cca510b0461648fe9c9ef63cb8839147, role_id=6005bd32383a4706a69108f7a2cf3ed1)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:36.196235 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9e6144-3917-437c-9729-b065c287e8be None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:36.214021 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 208/409] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Jun 29 12:20:35 2022] PUT /identity/v3/projects/9cd49b1ddb8042ec8620180f9db712a0/groups/cca510b0461648fe9c9ef63cb8839147/roles/6005bd32383a4706a69108f7a2cf3ed1 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:20:36.217225 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 202/410] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:36.243594 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e6144-3917-437c-9729-b065c287e8be None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:36.363550 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:36.411101 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e6144-3917-437c-9729-b065c287e8be None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:36.455462 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2173110-8eb3-49ab-ade1-0ef5ccb1558c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:36.456263 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2173110-8eb3-49ab-ade1-0ef5ccb1558c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:36.456546 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2173110-8eb3-49ab-ade1-0ef5ccb1558c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:36.456822 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2173110-8eb3-49ab-ade1-0ef5ccb1558c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:36.468188 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2173110-8eb3-49ab-ade1-0ef5ccb1558c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:36.475859 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b2173110-8eb3-49ab-ade1-0ef5ccb1558c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:20:36.475859 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:36.475859 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:36.475859 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:36.475859 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:36.475859 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:36.475859 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:36.475859 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:36.475859 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:36.475859 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:36.475859 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:36.475859 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:36.475859 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:36.475859 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:36.475859 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:36.475859 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:36.476751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:36.476751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:36.476751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:36.476751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:36.476751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:36.476751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:36.476751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:36.476751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:36.476751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:36.476751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:36.476751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:36.476751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:36.476751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:36.476751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:36.476751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:36.476751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:36.477633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:36.477633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:36.477633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:36.477633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:36.477633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:36.477633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:36.477633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:36.477633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:36.477633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:36.477633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:36.477633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:20:36.477633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:36.477633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 209/411] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:20:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:36.484424 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c75232-ec74-4964-8bf7-36b09de3004d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:36.495833 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401864f-8b09-4e03-a00e-edf706680cba None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:36.496370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401864f-8b09-4e03-a00e-edf706680cba None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:36.496650 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401864f-8b09-4e03-a00e-edf706680cba None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:36.498161 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 210/412] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:36.503782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9338d5-374f-4dd8-b46a-ae202d7c32f5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:36.504005 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9338d5-374f-4dd8-b46a-ae202d7c32f5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:36.504269 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9338d5-374f-4dd8-b46a-ae202d7c32f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:36.527767 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3a9338d5-374f-4dd8-b46a-ae202d7c32f5 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:36.528960 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3a9338d5-374f-4dd8-b46a-ae202d7c32f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:36.622265 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:36.662126 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 211/413] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:20:36.671810 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c75232-ec74-4964-8bf7-36b09de3004d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:36.674754 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09564be-851a-432a-a796-253fdcfb6ea1 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:36.674754 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09564be-851a-432a-a796-253fdcfb6ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:36.674754 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09564be-851a-432a-a796-253fdcfb6ea1 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:36.676326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 212/414] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:20:36.680956 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99db24e-0300-4896-bb06-ba37bb93a5ad None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:36.681298 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99db24e-0300-4896-bb06-ba37bb93a5ad None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:36.681582 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99db24e-0300-4896-bb06-ba37bb93a5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:36.703307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e99db24e-0300-4896-bb06-ba37bb93a5ad None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:36.704572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e99db24e-0300-4896-bb06-ba37bb93a5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:36.712517 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41508d4c-a702-4377-8d17-eb49b66a989f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:36.713531 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41508d4c-a702-4377-8d17-eb49b66a989f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:36.714121 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41508d4c-a702-4377-8d17-eb49b66a989f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:36.714758 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41508d4c-a702-4377-8d17-eb49b66a989f None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:36.715863 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41508d4c-a702-4377-8d17-eb49b66a989f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:36.716369 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41508d4c-a702-4377-8d17-eb49b66a989f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:36.717947 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41508d4c-a702-4377-8d17-eb49b66a989f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:36.727431 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 203/415] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:36.728760 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41508d4c-a702-4377-8d17-eb49b66a989f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:36.755826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:36.782443 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 213/416] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:36.782443 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99db24e-0300-4896-bb06-ba37bb93a5ad None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:36.869160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41508d4c-a702-4377-8d17-eb49b66a989f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:36.897100 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99db24e-0300-4896-bb06-ba37bb93a5ad None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:36.900622 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:36.908711 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b782ac1a-eb01-49f1-914c-7cd4e395e7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:36.909712 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782ac1a-eb01-49f1-914c-7cd4e395e7f2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:36.910139 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782ac1a-eb01-49f1-914c-7cd4e395e7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:36.910588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b782ac1a-eb01-49f1-914c-7cd4e395e7f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:36.923025 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b782ac1a-eb01-49f1-914c-7cd4e395e7f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:36.931326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b782ac1a-eb01-49f1-914c-7cd4e395e7f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:20:36.931326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:36.931326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:36.931326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:36.931326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:36.931326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:36.931326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:36.931326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:36.931326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:36.931326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:36.931326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:36.931326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:36.931326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:36.931326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:36.931326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:36.931326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:36.932518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:36.932518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:36.932518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:36.932518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:36.932518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:36.932518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:36.932518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:36.932518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:36.932518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:36.932518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:36.932518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:36.932518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:36.932518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:36.932518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:36.932518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:36.932518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:36.933577 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:36.933577 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:36.933577 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:36.933577 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:36.933577 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:36.933577 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:36.933577 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:36.933577 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:36.933577 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:36.933577 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:36.933577 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:20:36.933577 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:36.934173 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d263b50e-b0e2-494f-90a3-875f9ab87e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:36.934173 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263b50e-b0e2-494f-90a3-875f9ab87e44 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:36.934173 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263b50e-b0e2-494f-90a3-875f9ab87e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:36.934173 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263b50e-b0e2-494f-90a3-875f9ab87e44 None admin] PATH_INFO: `/v3/groups/21405215d71d47b9b7884dde246ff5e1` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:36.934977 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d263b50e-b0e2-494f-90a3-875f9ab87e44 None admin] RBAC: Authorizing `identity:get_group(group_id=21405215d71d47b9b7884dde246ff5e1)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:36.936166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 204/417] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:36.936407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d263b50e-b0e2-494f-90a3-875f9ab87e44 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:36.937251 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b782ac1a-eb01-49f1-914c-7cd4e395e7f2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:36.938388 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 214/418] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 12:20:36 2022] GET /identity/v3/groups/21405215d71d47b9b7884dde246ff5e1 => generated 229 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:36.945919 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d263b50e-b0e2-494f-90a3-875f9ab87e44 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:37.001490 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:37.054979 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d263b50e-b0e2-494f-90a3-875f9ab87e44 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:37.055985 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b782ac1a-eb01-49f1-914c-7cd4e395e7f2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:37.093023 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49dd8a-29ad-4565-9c8e-6435cdcf65fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:37.093884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49dd8a-29ad-4565-9c8e-6435cdcf65fd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:37.094163 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49dd8a-29ad-4565-9c8e-6435cdcf65fd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:37.094453 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49dd8a-29ad-4565-9c8e-6435cdcf65fd None admin] PATH_INFO: `/v3/projects/a19795d69941402bbd95f8c9172fb339` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:37.094716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e394878-ba69-4e6f-b84a-97e710ec353c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:37.095680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49dd8a-29ad-4565-9c8e-6435cdcf65fd None admin] RBAC: Authorizing `identity:get_project(project_id=a19795d69941402bbd95f8c9172fb339)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:37.095792 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e394878-ba69-4e6f-b84a-97e710ec353c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:37.096191 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e394878-ba69-4e6f-b84a-97e710ec353c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:37.096600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e394878-ba69-4e6f-b84a-97e710ec353c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:37.097181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49dd8a-29ad-4565-9c8e-6435cdcf65fd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:37.099694 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 215/419] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:36 2022] GET /identity/v3/projects/a19795d69941402bbd95f8c9172fb339 => generated 344 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:37.100490 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49dd8a-29ad-4565-9c8e-6435cdcf65fd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:37.108421 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e394878-ba69-4e6f-b84a-97e710ec353c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:37.117354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2e394878-ba69-4e6f-b84a-97e710ec353c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 12:20:37.117354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:37.117354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:37.117354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:37.117354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:37.117354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:37.117354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:37.117354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:37.117354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:37.117354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:37.117354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:37.117354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:37.117354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:37.117354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:37.117354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:37.117354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:37.118443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:37.118443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:37.118443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:37.118443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:37.118443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:37.118443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:37.118443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:37.118443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:37.118443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:37.118443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:37.118443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:37.118443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:37.118443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:37.118443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:37.118443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:37.118443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:37.119414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:37.119414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:37.119414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:37.119414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:37.119414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:37.119414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:37.119414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:37.119414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:37.119414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:37.119414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:37.119414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 12:20:37.119414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:37.122298 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 205/420] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 12:20:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:37.123362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e394878-ba69-4e6f-b84a-97e710ec353c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:37.177396 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:37.208193 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49dd8a-29ad-4565-9c8e-6435cdcf65fd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:37.230838 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:37.234115 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e394878-ba69-4e6f-b84a-97e710ec353c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:37.246123 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821307c-ddda-4063-898a-146368878e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:37.247118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821307c-ddda-4063-898a-146368878e1b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:37.247227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821307c-ddda-4063-898a-146368878e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:37.247459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821307c-ddda-4063-898a-146368878e1b None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:37.248204 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821307c-ddda-4063-898a-146368878e1b None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:37.248535 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821307c-ddda-4063-898a-146368878e1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:37.249873 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0821307c-ddda-4063-898a-146368878e1b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:37.262636 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 216/421] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 12:20:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:37.262636 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821307c-ddda-4063-898a-146368878e1b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:37.279913 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd305c-ea85-4c99-b81e-ffa73e5ac6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:37.281184 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd305c-ea85-4c99-b81e-ffa73e5ac6ec None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:37.281709 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd305c-ea85-4c99-b81e-ffa73e5ac6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:37.282178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd305c-ea85-4c99-b81e-ffa73e5ac6ec None admin] PATH_INFO: `/v3/projects/a19795d69941402bbd95f8c9172fb339/groups/21405215d71d47b9b7884dde246ff5e1/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:37.284495 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cd305c-ea85-4c99-b81e-ffa73e5ac6ec None admin] RBAC: Authorizing `identity:create_grant(project_id=a19795d69941402bbd95f8c9172fb339, group_id=21405215d71d47b9b7884dde246ff5e1, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:37.286145 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cd305c-ea85-4c99-b81e-ffa73e5ac6ec None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:37.300726 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 206/422] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Jun 29 12:20:37 2022] PUT /identity/v3/projects/a19795d69941402bbd95f8c9172fb339/groups/21405215d71d47b9b7884dde246ff5e1/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:20:37.301879 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd305c-ea85-4c99-b81e-ffa73e5ac6ec None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:37.374691 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0821307c-ddda-4063-898a-146368878e1b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:37.413496 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1bd9c4-f232-4dc4-a2a3-bc9eeda2ed70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:37.414194 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1bd9c4-f232-4dc4-a2a3-bc9eeda2ed70 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:37.414514 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1bd9c4-f232-4dc4-a2a3-bc9eeda2ed70 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:37.414802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1bd9c4-f232-4dc4-a2a3-bc9eeda2ed70 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:37.415674 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1bd9c4-f232-4dc4-a2a3-bc9eeda2ed70 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:37.415996 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1bd9c4-f232-4dc4-a2a3-bc9eeda2ed70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:37.418044 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1bd9c4-f232-4dc4-a2a3-bc9eeda2ed70 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:37.427426 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 217/423] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:20:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:37.433009 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1bd9c4-f232-4dc4-a2a3-bc9eeda2ed70 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:37.438984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd305c-ea85-4c99-b81e-ffa73e5ac6ec None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:37.460246 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:37.517122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c2cf1-3471-4aee-a96b-003ce2a401b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:37.517703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c2cf1-3471-4aee-a96b-003ce2a401b5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:37.517990 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c2cf1-3471-4aee-a96b-003ce2a401b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:37.518267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96c2cf1-3471-4aee-a96b-003ce2a401b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:37.534305 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:37.607058 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c2cf1-3471-4aee-a96b-003ce2a401b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:37.607372 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1bd9c4-f232-4dc4-a2a3-bc9eeda2ed70 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:37.608512 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96c2cf1-3471-4aee-a96b-003ce2a401b5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:37.667260 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476dbc5a-6732-4818-889d-8b06c0dc94d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:37.668099 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dbc5a-6732-4818-889d-8b06c0dc94d9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:37.668390 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dbc5a-6732-4818-889d-8b06c0dc94d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:37.668665 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dbc5a-6732-4818-889d-8b06c0dc94d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:37.723535 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 207/424] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:20:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:20:37.735509 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c2cf1-3471-4aee-a96b-003ce2a401b5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:37.759352 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:37.814186 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476dbc5a-6732-4818-889d-8b06c0dc94d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:37.815653 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476dbc5a-6732-4818-889d-8b06c0dc94d9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:37.863918 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:37.922810 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 218/425] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:20:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 12:20:37.924625 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476dbc5a-6732-4818-889d-8b06c0dc94d9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:37.957581 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96c2cf1-3471-4aee-a96b-003ce2a401b5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:38.012834 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8bb56-bbca-4390-a40a-ceadc736dac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:38.013701 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8bb56-bbca-4390-a40a-ceadc736dac0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:38.013989 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8bb56-bbca-4390-a40a-ceadc736dac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:38.014265 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8bb56-bbca-4390-a40a-ceadc736dac0 None admin] PATH_INFO: `/v3/projects/a19c893c70c84649bb5406d2cf9e2708` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:38.015551 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b8bb56-bbca-4390-a40a-ceadc736dac0 None admin] RBAC: Authorizing `identity:get_project(project_id=a19c893c70c84649bb5406d2cf9e2708)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:38.016995 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b8bb56-bbca-4390-a40a-ceadc736dac0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:38.019016 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 208/426] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:37 2022] GET /identity/v3/projects/a19c893c70c84649bb5406d2cf9e2708 => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:38.028867 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8bb56-bbca-4390-a40a-ceadc736dac0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:38.056648 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476dbc5a-6732-4818-889d-8b06c0dc94d9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:38.099551 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3149a13-d8d9-48d1-a307-c2a722027c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:38.100430 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3149a13-d8d9-48d1-a307-c2a722027c80 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:38.100725 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3149a13-d8d9-48d1-a307-c2a722027c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:38.101008 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3149a13-d8d9-48d1-a307-c2a722027c80 None admin] PATH_INFO: `/v3/projects/a19c893c70c84649bb5406d2cf9e2708` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:38.102223 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3149a13-d8d9-48d1-a307-c2a722027c80 None admin] RBAC: Authorizing `identity:get_project(project_id=a19c893c70c84649bb5406d2cf9e2708)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:38.103866 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3149a13-d8d9-48d1-a307-c2a722027c80 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:38.106095 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 219/427] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:37 2022] GET /identity/v3/projects/a19c893c70c84649bb5406d2cf9e2708 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:38.111157 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3149a13-d8d9-48d1-a307-c2a722027c80 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:38.129181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:38.206789 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:38.245005 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8bb56-bbca-4390-a40a-ceadc736dac0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:38.268901 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3149a13-d8d9-48d1-a307-c2a722027c80 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:38.309053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeead51e-e2e8-48f4-a2e8-df8b19141ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:38.309927 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeead51e-e2e8-48f4-a2e8-df8b19141ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:38.310265 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeead51e-e2e8-48f4-a2e8-df8b19141ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:38.310578 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeead51e-e2e8-48f4-a2e8-df8b19141ee3 None admin] PATH_INFO: `/v3/groups/cca510b0461648fe9c9ef63cb8839147` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:38.312436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeead51e-e2e8-48f4-a2e8-df8b19141ee3 None admin] RBAC: Authorizing `identity:get_group(group_id=cca510b0461648fe9c9ef63cb8839147)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:38.313885 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeead51e-e2e8-48f4-a2e8-df8b19141ee3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:38.315771 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 209/428] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 12:20:38 2022] GET /identity/v3/groups/cca510b0461648fe9c9ef63cb8839147 => generated 226 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:38.323362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeead51e-e2e8-48f4-a2e8-df8b19141ee3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:38.333539 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e9e52-b763-4379-bb4b-7874e0e4fe89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:38.334408 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e9e52-b763-4379-bb4b-7874e0e4fe89 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:38.334717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e9e52-b763-4379-bb4b-7874e0e4fe89 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:38.334997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e9e52-b763-4379-bb4b-7874e0e4fe89 None admin] PATH_INFO: `/v3/groups/cca510b0461648fe9c9ef63cb8839147` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:38.337593 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e9e52-b763-4379-bb4b-7874e0e4fe89 None admin] RBAC: Authorizing `identity:get_group(group_id=cca510b0461648fe9c9ef63cb8839147)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:38.339285 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e9e52-b763-4379-bb4b-7874e0e4fe89 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:38.343122 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 220/429] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 12:20:38 2022] GET /identity/v3/groups/cca510b0461648fe9c9ef63cb8839147 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:38.347611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e9e52-b763-4379-bb4b-7874e0e4fe89 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:38.495513 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:38.517475 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeead51e-e2e8-48f4-a2e8-df8b19141ee3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:38.533937 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e9e52-b763-4379-bb4b-7874e0e4fe89 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:38.568578 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50853193-7cea-4b7c-a312-bc407a6f647e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:38.569394 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50853193-7cea-4b7c-a312-bc407a6f647e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:38.569685 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50853193-7cea-4b7c-a312-bc407a6f647e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:38.569967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50853193-7cea-4b7c-a312-bc407a6f647e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:38.570846 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50853193-7cea-4b7c-a312-bc407a6f647e None admin] RBAC: Adding query filter params (group.id=cca510b0461648fe9c9ef63cb8839147, role.id=3bf034b395b44d88b3af67da01a6ed82, scope.project.id=a19c893c70c84649bb5406d2cf9e2708) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:38.571164 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50853193-7cea-4b7c-a312-bc407a6f647e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:38.572612 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50853193-7cea-4b7c-a312-bc407a6f647e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:38.580648 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 210/430] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jun 29 12:20:38 2022] GET /identity/v3/role_assignments?group.id=cca510b0461648fe9c9ef63cb8839147&scope.project.id=a19c893c70c84649bb5406d2cf9e2708&role.id=3bf034b395b44d88b3af67da01a6ed82 => generated 611 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:38.586769 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd04d0-082e-42b0-a792-50c3ef363801 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:38.587129 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd04d0-082e-42b0-a792-50c3ef363801 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:38.587399 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd04d0-082e-42b0-a792-50c3ef363801 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:38.591732 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 211/431] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:38 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:38.597904 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f81e4-5d05-4d71-904b-215a593e35f7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:38.598225 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f81e4-5d05-4d71-904b-215a593e35f7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:38.598503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3f81e4-5d05-4d71-904b-215a593e35f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:38.619612 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a52d4-7bc3-4ec0-8403-6df6379176d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:38.620750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a52d4-7bc3-4ec0-8403-6df6379176d0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:38.621144 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a52d4-7bc3-4ec0-8403-6df6379176d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:38.621452 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a52d4-7bc3-4ec0-8403-6df6379176d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:38.622364 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38a52d4-7bc3-4ec0-8403-6df6379176d0 None admin] RBAC: Adding query filter params (group.id=cca510b0461648fe9c9ef63cb8839147, role.id=6005bd32383a4706a69108f7a2cf3ed1, scope.project.id=a19c893c70c84649bb5406d2cf9e2708) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:38.622808 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38a52d4-7bc3-4ec0-8403-6df6379176d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:38.624397 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38a52d4-7bc3-4ec0-8403-6df6379176d0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:38.633824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7b3f81e4-5d05-4d71-904b-215a593e35f7 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:38.635071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7b3f81e4-5d05-4d71-904b-215a593e35f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:38.638391 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:38.638836 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 221/432] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jun 29 12:20:38 2022] GET /identity/v3/role_assignments?group.id=cca510b0461648fe9c9ef63cb8839147&scope.project.id=a19c893c70c84649bb5406d2cf9e2708&role.id=6005bd32383a4706a69108f7a2cf3ed1 => generated 611 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:38.640212 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2705f9c-ab92-47b0-bb56-4f61d3dc3cbe None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:38.640559 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2705f9c-ab92-47b0-bb56-4f61d3dc3cbe None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:38.640851 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2705f9c-ab92-47b0-bb56-4f61d3dc3cbe None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:38.642122 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 222/433] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:38.648029 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b9104-437a-4f71-954d-053fa0cb978b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:38.648336 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b9104-437a-4f71-954d-053fa0cb978b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:38.648599 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b9104-437a-4f71-954d-053fa0cb978b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:38.667404 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5c6b9104-437a-4f71-954d-053fa0cb978b None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:38.668641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5c6b9104-437a-4f71-954d-053fa0cb978b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:38.741380 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 212/434] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:38.805053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 223/435] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:20:38.808222 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f81e4-5d05-4d71-904b-215a593e35f7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:38.833146 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b9104-437a-4f71-954d-053fa0cb978b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:38.961923 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:38.987878 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3f81e4-5d05-4d71-904b-215a593e35f7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:38.995798 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b9104-437a-4f71-954d-053fa0cb978b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:39.030852 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc91e3-25e0-49c1-87a3-3b2579e1052b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:39.030852 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc91e3-25e0-49c1-87a3-3b2579e1052b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:39.030852 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc91e3-25e0-49c1-87a3-3b2579e1052b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:39.031195 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc91e3-25e0-49c1-87a3-3b2579e1052b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:39.042086 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8bd41-2ad6-4cf9-a7a2-ac325c875b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:39.042953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8bd41-2ad6-4cf9-a7a2-ac325c875b3c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:39.043247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8bd41-2ad6-4cf9-a7a2-ac325c875b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:39.043530 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8bd41-2ad6-4cf9-a7a2-ac325c875b3c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:39.048095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:39.056323 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc91e3-25e0-49c1-87a3-3b2579e1052b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:39.060117 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8bd41-2ad6-4cf9-a7a2-ac325c875b3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:39.069088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-67f8bd41-2ad6-4cf9-a7a2-ac325c875b3c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 12:20:39.069088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:39.069088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:39.069088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:39.069088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:39.069088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:39.069088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:39.069088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:39.069088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:39.069088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:39.069088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:39.069088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:39.069088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:39.069088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:39.069088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:39.069088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:39.069995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:39.069995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:39.069995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:39.069995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:39.069995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:39.069995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:39.069995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:39.069995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:39.069995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:39.069995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:39.069995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:39.069995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:39.069995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:39.069995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:39.069995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:39.069995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:39.074180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:39.074180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:39.074180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:39.074180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:39.074180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:39.074180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:39.074180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:39.074180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:39.074180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:39.074180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:39.074180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 12:20:39.074180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:39.074180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 224/436] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 12:20:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:39.074945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-92cc91e3-25e0-49c1-87a3-3b2579e1052b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:20:39.074945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:39.074945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:39.074945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:39.074945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:39.074945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:39.074945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:39.074945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:39.074945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:39.074945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:39.074945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:39.074945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:39.074945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:39.074945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:39.074945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:39.074945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:39.075768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:39.075768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:39.075768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:39.075768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:39.075768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:39.075768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:39.075768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:39.075768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:39.075768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:39.075768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:39.075768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:39.075768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:39.075768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:39.075768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:39.075768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:39.075768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:39.076563 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:39.076563 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:39.076563 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:39.076563 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:39.076563 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:39.076563 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:39.076563 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:39.076563 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:39.076563 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:39.076563 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:39.076563 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 29 12:20:39.076563 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:39.076563 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 213/437] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:39.078456 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8bd41-2ad6-4cf9-a7a2-ac325c875b3c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:39.086117 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc91e3-25e0-49c1-87a3-3b2579e1052b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:39.231001 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8bd41-2ad6-4cf9-a7a2-ac325c875b3c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:39.231398 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc91e3-25e0-49c1-87a3-3b2579e1052b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:39.283017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a989425-404f-4a21-89e1-baa2cd61ebd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:39.283788 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a989425-404f-4a21-89e1-baa2cd61ebd0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:39.284076 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a989425-404f-4a21-89e1-baa2cd61ebd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:39.284357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a989425-404f-4a21-89e1-baa2cd61ebd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:39.285107 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a989425-404f-4a21-89e1-baa2cd61ebd0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:39.285419 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a989425-404f-4a21-89e1-baa2cd61ebd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:39.287020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a989425-404f-4a21-89e1-baa2cd61ebd0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:39.287863 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e550906-9dd1-4b75-bb7a-1a1df6132323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:39.288989 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e550906-9dd1-4b75-bb7a-1a1df6132323 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:39.289829 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e550906-9dd1-4b75-bb7a-1a1df6132323 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:39.290244 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e550906-9dd1-4b75-bb7a-1a1df6132323 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:39.291556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e550906-9dd1-4b75-bb7a-1a1df6132323 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:39.292088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e550906-9dd1-4b75-bb7a-1a1df6132323 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:39.294645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e550906-9dd1-4b75-bb7a-1a1df6132323 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:39.297646 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:39.299227 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 214/438] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 12:20:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:39.304577 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a989425-404f-4a21-89e1-baa2cd61ebd0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:39.308743 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 225/439] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:20:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:39.316241 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e550906-9dd1-4b75-bb7a-1a1df6132323 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:39.412565 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:39.455263 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a989425-404f-4a21-89e1-baa2cd61ebd0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:39.459807 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e550906-9dd1-4b75-bb7a-1a1df6132323 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:39.494232 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b9b63-422b-4960-958d-c802816f704f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:39.495700 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b9b63-422b-4960-958d-c802816f704f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:39.496015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b9b63-422b-4960-958d-c802816f704f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:39.496637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b9b63-422b-4960-958d-c802816f704f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:39.508274 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1ce76-6125-4b06-bbb3-126537cfdf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:39.508274 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1ce76-6125-4b06-bbb3-126537cfdf30 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:39.508274 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1ce76-6125-4b06-bbb3-126537cfdf30 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:39.508274 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1ce76-6125-4b06-bbb3-126537cfdf30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:39.574520 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:39.575953 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b9b63-422b-4960-958d-c802816f704f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:39.577393 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b9b63-422b-4960-958d-c802816f704f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:39.582778 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa1ce76-6125-4b06-bbb3-126537cfdf30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:39.584278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa1ce76-6125-4b06-bbb3-126537cfdf30 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:39.640883 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:39.690106 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 215/440] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:20:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 12:20:39.690291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 226/441] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:20:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 12:20:39.694118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1ce76-6125-4b06-bbb3-126537cfdf30 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:39.694322 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b9b63-422b-4960-958d-c802816f704f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:39.789012 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:39.796254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1ce76-6125-4b06-bbb3-126537cfdf30 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:39.799144 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b9b63-422b-4960-958d-c802816f704f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:39.829519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf2af8-c49c-47fc-ba81-03d4893744fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:39.830380 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf2af8-c49c-47fc-ba81-03d4893744fd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:39.830700 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf2af8-c49c-47fc-ba81-03d4893744fd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:39.830983 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf2af8-c49c-47fc-ba81-03d4893744fd None admin] PATH_INFO: `/v3/projects/9cd49b1ddb8042ec8620180f9db712a0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:39.832164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf2af8-c49c-47fc-ba81-03d4893744fd None admin] RBAC: Authorizing `identity:get_project(project_id=9cd49b1ddb8042ec8620180f9db712a0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:39.833595 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d621ee3-2ca7-431c-9cc7-3cf34a5a8156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:39.833728 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf2af8-c49c-47fc-ba81-03d4893744fd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:39.834483 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d621ee3-2ca7-431c-9cc7-3cf34a5a8156 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:39.834770 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d621ee3-2ca7-431c-9cc7-3cf34a5a8156 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:39.835045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d621ee3-2ca7-431c-9cc7-3cf34a5a8156 None admin] PATH_INFO: `/v3/projects/9cd49b1ddb8042ec8620180f9db712a0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:39.835357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 227/442] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:39 2022] GET /identity/v3/projects/9cd49b1ddb8042ec8620180f9db712a0 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:39.836521 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbda5d-d8f8-4f7e-8dca-39e7646ed12f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:39.836808 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbda5d-d8f8-4f7e-8dca-39e7646ed12f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:39.837086 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbda5d-d8f8-4f7e-8dca-39e7646ed12f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:39.838304 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 228/443] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:39.841364 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fbda5d-d8f8-4f7e-8dca-39e7646ed12f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:39.841497 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d621ee3-2ca7-431c-9cc7-3cf34a5a8156 None admin] RBAC: Authorizing `identity:get_project(project_id=9cd49b1ddb8042ec8620180f9db712a0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:39.842938 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d621ee3-2ca7-431c-9cc7-3cf34a5a8156 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:39.848963 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 216/444] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:39 2022] GET /identity/v3/projects/9cd49b1ddb8042ec8620180f9db712a0 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:39.848963 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a88a4-ecb5-40e8-ad54-d6bdef096211 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:39.848963 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a88a4-ecb5-40e8-ad54-d6bdef096211 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:39.848963 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a88a4-ecb5-40e8-ad54-d6bdef096211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:39.864045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3c2a88a4-ecb5-40e8-ad54-d6bdef096211 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:39.865249 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3c2a88a4-ecb5-40e8-ad54-d6bdef096211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:39.889487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:39.932888 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 217/445] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:39.933561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a88a4-ecb5-40e8-ad54-d6bdef096211 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:39.943008 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fbda5d-d8f8-4f7e-8dca-39e7646ed12f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:39.977545 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5821b0-b318-4444-9a1e-06d7af59ef0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:39.978400 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5821b0-b318-4444-9a1e-06d7af59ef0d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:39.978719 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5821b0-b318-4444-9a1e-06d7af59ef0d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:39.979487 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5821b0-b318-4444-9a1e-06d7af59ef0d None admin] PATH_INFO: `/v3/groups/cca510b0461648fe9c9ef63cb8839147` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:39.980787 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5821b0-b318-4444-9a1e-06d7af59ef0d None admin] RBAC: Authorizing `identity:get_group(group_id=cca510b0461648fe9c9ef63cb8839147)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:39.982237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5821b0-b318-4444-9a1e-06d7af59ef0d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:39.983901 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 229/446] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 12:20:39 2022] GET /identity/v3/groups/cca510b0461648fe9c9ef63cb8839147 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:39.984663 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5821b0-b318-4444-9a1e-06d7af59ef0d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:40.015005 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:40.048982 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a88a4-ecb5-40e8-ad54-d6bdef096211 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:40.082781 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d862a-c9b5-4401-ad2b-1fe28d56264a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:40.083643 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d862a-c9b5-4401-ad2b-1fe28d56264a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:40.083800 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5821b0-b318-4444-9a1e-06d7af59ef0d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:40.083930 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d862a-c9b5-4401-ad2b-1fe28d56264a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:40.084211 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d862a-c9b5-4401-ad2b-1fe28d56264a None admin] PATH_INFO: `/v3/groups/cca510b0461648fe9c9ef63cb8839147` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:40.085391 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d862a-c9b5-4401-ad2b-1fe28d56264a None admin] RBAC: Authorizing `identity:get_group(group_id=cca510b0461648fe9c9ef63cb8839147)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:40.086900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d862a-c9b5-4401-ad2b-1fe28d56264a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:40.089068 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 218/447] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 12:20:39 2022] GET /identity/v3/groups/cca510b0461648fe9c9ef63cb8839147 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:40.089838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d862a-c9b5-4401-ad2b-1fe28d56264a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:40.117975 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb301d-539c-4c55-b276-48422a174b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:40.118801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb301d-539c-4c55-b276-48422a174b2d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:40.119087 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb301d-539c-4c55-b276-48422a174b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:40.119370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb301d-539c-4c55-b276-48422a174b2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:40.120825 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:40.129556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb301d-539c-4c55-b276-48422a174b2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:40.137517 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-43cb301d-539c-4c55-b276-48422a174b2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:20:40.137517 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:40.137517 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:40.137517 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:40.137517 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:40.137517 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:40.137517 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:40.137517 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:40.137517 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:40.137517 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:40.137517 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:40.137517 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:40.137517 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:40.137517 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:40.137517 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:40.137517 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:40.138513 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:40.138513 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:40.138513 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:40.138513 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:40.138513 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:40.138513 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:40.138513 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:40.138513 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:40.138513 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:40.138513 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:40.138513 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:40.138513 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:40.138513 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:40.138513 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:40.138513 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:40.138513 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:40.139326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:40.139326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:40.139326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:40.139326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:40.139326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:40.139326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:40.139326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:40.139326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:40.139326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:40.139326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:40.139326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:20:40.139326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:40.139326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 230/448] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:20:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:40.140029 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb301d-539c-4c55-b276-48422a174b2d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:40.197878 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d862a-c9b5-4401-ad2b-1fe28d56264a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:40.231737 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:40.233282 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10fe8f6-78b4-4640-80a6-4b35c06fe401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:40.233983 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fe8f6-78b4-4640-80a6-4b35c06fe401 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:40.234195 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fe8f6-78b4-4640-80a6-4b35c06fe401 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:40.234483 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fe8f6-78b4-4640-80a6-4b35c06fe401 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:40.235311 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10fe8f6-78b4-4640-80a6-4b35c06fe401 None admin] RBAC: Adding query filter params (group.id=cca510b0461648fe9c9ef63cb8839147, role.id=6005bd32383a4706a69108f7a2cf3ed1, scope.project.id=9cd49b1ddb8042ec8620180f9db712a0) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:40.235616 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10fe8f6-78b4-4640-80a6-4b35c06fe401 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:40.237051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10fe8f6-78b4-4640-80a6-4b35c06fe401 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:40.240789 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb301d-539c-4c55-b276-48422a174b2d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:40.242798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 219/449] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jun 29 12:20:40 2022] GET /identity/v3/role_assignments?group.id=cca510b0461648fe9c9ef63cb8839147&scope.project.id=9cd49b1ddb8042ec8620180f9db712a0&role.id=6005bd32383a4706a69108f7a2cf3ed1 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:40.243567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10fe8f6-78b4-4640-80a6-4b35c06fe401 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:40.279964 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a25a7-befd-42f6-b118-c35145bc6641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:40.280649 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a25a7-befd-42f6-b118-c35145bc6641 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:40.280936 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a25a7-befd-42f6-b118-c35145bc6641 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:40.281222 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a25a7-befd-42f6-b118-c35145bc6641 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:40.282198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a25a7-befd-42f6-b118-c35145bc6641 None admin] RBAC: Adding query filter params (group.id=cca510b0461648fe9c9ef63cb8839147, role.id=3bf034b395b44d88b3af67da01a6ed82, scope.project.id=9cd49b1ddb8042ec8620180f9db712a0) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:40.282583 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a25a7-befd-42f6-b118-c35145bc6641 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:40.284160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a25a7-befd-42f6-b118-c35145bc6641 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:40.302544 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 231/450] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jun 29 12:20:40 2022] GET /identity/v3/role_assignments?group.id=cca510b0461648fe9c9ef63cb8839147&scope.project.id=9cd49b1ddb8042ec8620180f9db712a0&role.id=3bf034b395b44d88b3af67da01a6ed82 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:40.375077 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:40.386538 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10fe8f6-78b4-4640-80a6-4b35c06fe401 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:40.428978 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b2d2db-ad58-4bd5-ae1c-adc4a89324c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:40.429673 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2d2db-ad58-4bd5-ae1c-adc4a89324c9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:40.429962 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2d2db-ad58-4bd5-ae1c-adc4a89324c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:40.430262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2d2db-ad58-4bd5-ae1c-adc4a89324c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:40.431048 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b2d2db-ad58-4bd5-ae1c-adc4a89324c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:40.431362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b2d2db-ad58-4bd5-ae1c-adc4a89324c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:40.432854 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b2d2db-ad58-4bd5-ae1c-adc4a89324c9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:40.442789 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 220/451] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:40.449777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a25a7-befd-42f6-b118-c35145bc6641 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:40.552803 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a25a7-befd-42f6-b118-c35145bc6641 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:40.586555 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99061cac-b73c-4a0a-9b0a-58b180fb24fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:40.587202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99061cac-b73c-4a0a-9b0a-58b180fb24fa None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:40.587483 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99061cac-b73c-4a0a-9b0a-58b180fb24fa None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:40.587760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99061cac-b73c-4a0a-9b0a-58b180fb24fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:40.613367 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:40.657206 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99061cac-b73c-4a0a-9b0a-58b180fb24fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:40.658706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99061cac-b73c-4a0a-9b0a-58b180fb24fa None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:40.758237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 232/452] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:20:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:20:40.764514 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b2d2db-ad58-4bd5-ae1c-adc4a89324c9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:40.863769 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b2d2db-ad58-4bd5-ae1c-adc4a89324c9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:40.896921 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a89f9ec-f0ac-4b72-9d24-5e226b786fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:40.897777 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89f9ec-f0ac-4b72-9d24-5e226b786fae None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:40.898063 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89f9ec-f0ac-4b72-9d24-5e226b786fae None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:40.898344 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89f9ec-f0ac-4b72-9d24-5e226b786fae None admin] PATH_INFO: `/v3/projects/a19795d69941402bbd95f8c9172fb339` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:40.899553 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a89f9ec-f0ac-4b72-9d24-5e226b786fae None admin] RBAC: Authorizing `identity:get_project(project_id=a19795d69941402bbd95f8c9172fb339)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:40.901037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a89f9ec-f0ac-4b72-9d24-5e226b786fae None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:40.902858 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 221/453] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:20:40 2022] GET /identity/v3/projects/a19795d69941402bbd95f8c9172fb339 => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:40.908253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99061cac-b73c-4a0a-9b0a-58b180fb24fa None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:40.962596 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:41.004160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99061cac-b73c-4a0a-9b0a-58b180fb24fa None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:41.036945 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ec4b1-e050-483f-949a-2696a234a30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:41.037781 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ec4b1-e050-483f-949a-2696a234a30b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:41.038060 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ec4b1-e050-483f-949a-2696a234a30b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:41.038334 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ec4b1-e050-483f-949a-2696a234a30b None admin] PATH_INFO: `/v3/groups/21405215d71d47b9b7884dde246ff5e1` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:41.039506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ec4b1-e050-483f-949a-2696a234a30b None admin] RBAC: Authorizing `identity:get_group(group_id=21405215d71d47b9b7884dde246ff5e1)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:41.040931 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ec4b1-e050-483f-949a-2696a234a30b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:41.042637 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 233/454] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jun 29 12:20:40 2022] GET /identity/v3/groups/21405215d71d47b9b7884dde246ff5e1 => generated 229 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:41.048317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a89f9ec-f0ac-4b72-9d24-5e226b786fae None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:41.074053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:41.145470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a89f9ec-f0ac-4b72-9d24-5e226b786fae None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:41.177524 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54920043-e5d5-47e6-b960-9f834669f6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:41.178162 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54920043-e5d5-47e6-b960-9f834669f6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:41.178469 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54920043-e5d5-47e6-b960-9f834669f6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:41.178753 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54920043-e5d5-47e6-b960-9f834669f6b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:41.179578 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54920043-e5d5-47e6-b960-9f834669f6b0 None admin] RBAC: Adding query filter params (group.id=21405215d71d47b9b7884dde246ff5e1, role.id=3a7f4bf556344407a6d9540c89d13641, scope.project.id=a19795d69941402bbd95f8c9172fb339) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:41.179888 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54920043-e5d5-47e6-b960-9f834669f6b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:41.181299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54920043-e5d5-47e6-b960-9f834669f6b0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:41.186966 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 222/455] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jun 29 12:20:41 2022] GET /identity/v3/role_assignments?group.id=21405215d71d47b9b7884dde246ff5e1&scope.project.id=a19795d69941402bbd95f8c9172fb339&role.id=3a7f4bf556344407a6d9540c89d13641 => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:43.705903 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380aac1-7422-4f31-9f0a-89f022fb7f13 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:43.706398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380aac1-7422-4f31-9f0a-89f022fb7f13 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:43.706398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380aac1-7422-4f31-9f0a-89f022fb7f13 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:43.708006 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 234/456] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:43.724239 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb9c35-d9ed-43e1-99ec-c21cf942e49b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:43.725239 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb9c35-d9ed-43e1-99ec-c21cf942e49b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:43.725714 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdb9c35-d9ed-43e1-99ec-c21cf942e49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:43.747430 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5fdb9c35-d9ed-43e1-99ec-c21cf942e49b None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:43.748943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5fdb9c35-d9ed-43e1-99ec-c21cf942e49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:43.841222 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 223/457] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:43.878937 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380aac1-7422-4f31-9f0a-89f022fb7f13 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:43.894368 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26892e73-76d2-4027-b766-c4a9b0783648 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:43.894724 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26892e73-76d2-4027-b766-c4a9b0783648 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:43.895007 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26892e73-76d2-4027-b766-c4a9b0783648 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:43.897560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 224/458] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:20:43.903433 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc167253-7382-4cc0-b4c8-f1488925b0b3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:43.903968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc167253-7382-4cc0-b4c8-f1488925b0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:43.904438 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc167253-7382-4cc0-b4c8-f1488925b0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:43.931244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cc167253-7382-4cc0-b4c8-f1488925b0b3 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:43.934029 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cc167253-7382-4cc0-b4c8-f1488925b0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:43.958185 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:44.048232 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 225/459] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:20:44.049842 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9e9de-370f-483e-8826-18aaed1e6aad None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:44.050146 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380aac1-7422-4f31-9f0a-89f022fb7f13 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:44.050302 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9e9de-370f-483e-8826-18aaed1e6aad None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:44.050578 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade9e9de-370f-483e-8826-18aaed1e6aad None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:44.051822 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 226/460] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:44.057657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122d2c1-a2ee-40b1-abdd-7cbbebffb92d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:44.058301 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122d2c1-a2ee-40b1-abdd-7cbbebffb92d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:44.058536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122d2c1-a2ee-40b1-abdd-7cbbebffb92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:44.078041 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9122d2c1-a2ee-40b1-abdd-7cbbebffb92d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:44.078951 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:44.081522 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9122d2c1-a2ee-40b1-abdd-7cbbebffb92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:44.096663 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbb896-b670-4c3d-b102-acad63b3a255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:44.097544 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbb896-b670-4c3d-b102-acad63b3a255 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:44.097899 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbb896-b670-4c3d-b102-acad63b3a255 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:44.098256 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbbb896-b670-4c3d-b102-acad63b3a255 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:44.109270 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbbb896-b670-4c3d-b102-acad63b3a255 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:44.110886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbbb896-b670-4c3d-b102-acad63b3a255 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:44.129468 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-bfbbb896-b670-4c3d-b102-acad63b3a255 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:44.129468 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:44.129468 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:44.129468 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:44.129468 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:44.129468 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:44.129468 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:44.129468 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:44.129468 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:44.129468 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:44.129468 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:44.129468 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:44.129468 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:20:44.129468 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:20:44.129468 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:20:44.130370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:20:44.130370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:44.130370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:44.130370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:44.130370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:44.130370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:44.130370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:44.130370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:44.130370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:44.130370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:44.130370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:44.130370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:44.130370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:44.130370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:44.130370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:44.130370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:44.130370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:44.131261 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:20:44.131261 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:20:44.131261 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:44.131261 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:44.131261 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 235/461] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:44.136625 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbb896-b670-4c3d-b102-acad63b3a255 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:44.182730 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 227/462] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:44.182730 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122d2c1-a2ee-40b1-abdd-7cbbebffb92d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:44.283358 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbbb896-b670-4c3d-b102-acad63b3a255 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:44.288721 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:44.316538 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122d2c1-a2ee-40b1-abdd-7cbbebffb92d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:44.325656 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493fa6d-a6ec-45f3-8f3a-46e0c7144189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:44.326472 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493fa6d-a6ec-45f3-8f3a-46e0c7144189 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:44.326765 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493fa6d-a6ec-45f3-8f3a-46e0c7144189 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:44.327046 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493fa6d-a6ec-45f3-8f3a-46e0c7144189 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:44.337595 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b493fa6d-a6ec-45f3-8f3a-46e0c7144189 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:44.339120 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b493fa6d-a6ec-45f3-8f3a-46e0c7144189 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:44.357295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b493fa6d-a6ec-45f3-8f3a-46e0c7144189 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:44.357295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:44.357295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:44.357295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:44.357295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:44.357295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:44.357295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:44.357295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:44.357295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:44.357295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:44.357295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:44.357295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:44.357295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:20:44.357295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:20:44.357295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:20:44.358363 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:20:44.358363 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:44.358363 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:44.358363 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:44.358363 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:44.358363 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:44.358363 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:44.358363 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:44.358363 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:44.358363 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:44.358363 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:44.358363 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:44.358363 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:44.358363 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:44.358363 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:44.358363 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:44.358363 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:44.359337 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:20:44.359337 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:20:44.359337 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:44.359337 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:44.361340 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36662e1a-a77c-426c-bd43-aa1a4c742446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:44.362108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36662e1a-a77c-426c-bd43-aa1a4c742446 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:44.362583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36662e1a-a77c-426c-bd43-aa1a4c742446 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:44.363346 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36662e1a-a77c-426c-bd43-aa1a4c742446 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:44.363730 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36662e1a-a77c-426c-bd43-aa1a4c742446 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:44.364101 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36662e1a-a77c-426c-bd43-aa1a4c742446 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:44.365566 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36662e1a-a77c-426c-bd43-aa1a4c742446 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:44.366255 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 236/463] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:44.367141 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493fa6d-a6ec-45f3-8f3a-46e0c7144189 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:44.378611 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 228/464] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:44.379873 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2458f-5a4d-4ae7-8043-309c1a9f9d5d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:44.380167 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2458f-5a4d-4ae7-8043-309c1a9f9d5d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:44.380449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2458f-5a4d-4ae7-8043-309c1a9f9d5d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:44.381810 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 229/465] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:44.386532 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2458f-5a4d-4ae7-8043-309c1a9f9d5d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:44.403324 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:44.481549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493fa6d-a6ec-45f3-8f3a-46e0c7144189 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:44.492177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2458f-5a4d-4ae7-8043-309c1a9f9d5d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:44.516919 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23fd547-67a7-4beb-aad2-0d016d9b72ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:44.517706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fd547-67a7-4beb-aad2-0d016d9b72ff None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:44.517988 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fd547-67a7-4beb-aad2-0d016d9b72ff None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:44.518254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23fd547-67a7-4beb-aad2-0d016d9b72ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:44.528120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4688dab-9752-4a7d-b2b6-252d154836e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:44.528633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23fd547-67a7-4beb-aad2-0d016d9b72ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:44.528820 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4688dab-9752-4a7d-b2b6-252d154836e4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:44.529108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4688dab-9752-4a7d-b2b6-252d154836e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:44.529387 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4688dab-9752-4a7d-b2b6-252d154836e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:44.530078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23fd547-67a7-4beb-aad2-0d016d9b72ff None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:44.530180 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4688dab-9752-4a7d-b2b6-252d154836e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:44.530479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4688dab-9752-4a7d-b2b6-252d154836e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:44.531793 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4688dab-9752-4a7d-b2b6-252d154836e4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:44.534641 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:44.550615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 230/466] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:44.550615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4688dab-9752-4a7d-b2b6-252d154836e4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:44.550850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b23fd547-67a7-4beb-aad2-0d016d9b72ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:44.550850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:44.550850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:44.550850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:44.550850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:44.550850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:44.550850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:44.550850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:44.550850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:44.550850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:44.550850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:44.550850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:44.550850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:20:44.550850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:20:44.550850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:20:44.552295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:20:44.552295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:44.552295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:44.552295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:44.552295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:44.552295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:44.552295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:44.552295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:44.552295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:44.552295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:44.552295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:44.552295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:44.552295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:44.552295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:44.552295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:44.552295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:44.552295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:44.553155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:20:44.553155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:20:44.553155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:44.553155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:44.553155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 237/467] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:44.553155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316682f2-8a52-4a70-a954-c60ac792f17d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:44.553155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316682f2-8a52-4a70-a954-c60ac792f17d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:44.553155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316682f2-8a52-4a70-a954-c60ac792f17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:44.574928 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-316682f2-8a52-4a70-a954-c60ac792f17d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:44.576107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-316682f2-8a52-4a70-a954-c60ac792f17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:44.652555 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 238/468] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:44.655563 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316682f2-8a52-4a70-a954-c60ac792f17d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:44.663079 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4688dab-9752-4a7d-b2b6-252d154836e4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:44.671458 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:44.700290 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ebada-fa6a-4598-8870-2cde5a25e601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:44.701317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ebada-fa6a-4598-8870-2cde5a25e601 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:44.701806 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ebada-fa6a-4598-8870-2cde5a25e601 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:44.702289 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5ebada-fa6a-4598-8870-2cde5a25e601 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:44.703257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ebada-fa6a-4598-8870-2cde5a25e601 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:44.704898 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5ebada-fa6a-4598-8870-2cde5a25e601 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:44.727574 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 231/469] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 12:20:44 2022] POST /identity/v3/users => generated 246 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:20:44.729266 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ebada-fa6a-4598-8870-2cde5a25e601 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:44.779487 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316682f2-8a52-4a70-a954-c60ac792f17d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:44.801654 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:44.822527 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8664ea4-3a2f-4bca-af13-bd1fd9f91f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:44.824424 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8664ea4-3a2f-4bca-af13-bd1fd9f91f53 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:44.824713 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8664ea4-3a2f-4bca-af13-bd1fd9f91f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:44.825039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8664ea4-3a2f-4bca-af13-bd1fd9f91f53 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:44.825783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8664ea4-3a2f-4bca-af13-bd1fd9f91f53 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:44.827320 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8664ea4-3a2f-4bca-af13-bd1fd9f91f53 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:44.855004 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 239/470] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 12:20:44 2022] POST /identity/v3/users => generated 248 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:20:44.855975 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8664ea4-3a2f-4bca-af13-bd1fd9f91f53 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:44.883807 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5ebada-fa6a-4598-8870-2cde5a25e601 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:44.940166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2402041-aaac-4101-a324-2efaa3d85a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:44.941419 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2402041-aaac-4101-a324-2efaa3d85a72 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:44.941915 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2402041-aaac-4101-a324-2efaa3d85a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:44.944071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2402041-aaac-4101-a324-2efaa3d85a72 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:44.945910 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2402041-aaac-4101-a324-2efaa3d85a72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:44.946656 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2402041-aaac-4101-a324-2efaa3d85a72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:44.949506 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2402041-aaac-4101-a324-2efaa3d85a72 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:44.967106 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 232/471] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:44.973936 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2402041-aaac-4101-a324-2efaa3d85a72 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:44.994761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:45.051392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8664ea4-3a2f-4bca-af13-bd1fd9f91f53 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:45.106312 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03183d5c-3855-47b8-9bf5-93778ac82ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:45.107078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03183d5c-3855-47b8-9bf5-93778ac82ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:45.107364 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03183d5c-3855-47b8-9bf5-93778ac82ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:45.107644 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03183d5c-3855-47b8-9bf5-93778ac82ed0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:45.118738 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2402041-aaac-4101-a324-2efaa3d85a72 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:45.122107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03183d5c-3855-47b8-9bf5-93778ac82ed0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:45.123750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03183d5c-3855-47b8-9bf5-93778ac82ed0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:45.137847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:45.143042 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-03183d5c-3855-47b8-9bf5-93778ac82ed0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:45.143042 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:45.143042 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:45.143042 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:45.143042 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:45.143042 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:45.143042 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:45.143042 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:45.143042 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:45.143042 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:45.143042 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:45.143042 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:45.143042 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:20:45.143042 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:20:45.143042 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:20:45.143904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:20:45.143904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:45.143904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:45.143904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:45.143904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:45.143904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:45.143904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:45.143904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:45.143904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:45.143904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:45.143904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:45.143904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:45.143904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:45.143904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:45.143904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:45.143904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:45.143904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:45.144739 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:20:45.144739 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:20:45.144739 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:45.144739 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:45.144739 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 240/472] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:45.151790 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03183d5c-3855-47b8-9bf5-93778ac82ed0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:45.164466 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893f0dc9-5e80-4e17-862f-3ab1980e8d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:45.165160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f0dc9-5e80-4e17-862f-3ab1980e8d1f None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:45.165448 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f0dc9-5e80-4e17-862f-3ab1980e8d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:45.165734 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893f0dc9-5e80-4e17-862f-3ab1980e8d1f None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:45.167039 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893f0dc9-5e80-4e17-862f-3ab1980e8d1f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:45.168578 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893f0dc9-5e80-4e17-862f-3ab1980e8d1f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:45.196118 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 233/473] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 12:20:44 2022] POST /identity/v3/users => generated 248 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:20:45.234618 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bb7b0-99da-49b7-89cd-1af302a8cdf8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:45.234977 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bb7b0-99da-49b7-89cd-1af302a8cdf8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:45.235256 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bb7b0-99da-49b7-89cd-1af302a8cdf8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:45.237495 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 234/474] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:20:45.248045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a9165-22b6-42c4-81a4-e159c1e23f9c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:45.248385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a9165-22b6-42c4-81a4-e159c1e23f9c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:45.248612 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a9165-22b6-42c4-81a4-e159c1e23f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:45.282719 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03183d5c-3855-47b8-9bf5-93778ac82ed0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:45.303654 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0a5a9165-22b6-42c4-81a4-e159c1e23f9c None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:45.304921 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0a5a9165-22b6-42c4-81a4-e159c1e23f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:45.308971 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:45.354243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c736629-5870-45e9-9226-53c8e36081c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:45.354243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c736629-5870-45e9-9226-53c8e36081c9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:45.354243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c736629-5870-45e9-9226-53c8e36081c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:45.354243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c736629-5870-45e9-9226-53c8e36081c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:45.354886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c736629-5870-45e9-9226-53c8e36081c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:45.354886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c736629-5870-45e9-9226-53c8e36081c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:45.354886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c736629-5870-45e9-9226-53c8e36081c9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:45.396738 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 241/475] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:45.398051 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c736629-5870-45e9-9226-53c8e36081c9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:45.427641 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 235/476] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:45.451898 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a9165-22b6-42c4-81a4-e159c1e23f9c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:45.521667 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c736629-5870-45e9-9226-53c8e36081c9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:45.561799 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4e520-bd6f-404c-ba8f-7b57d286ba23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:45.562530 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4e520-bd6f-404c-ba8f-7b57d286ba23 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:45.562823 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4e520-bd6f-404c-ba8f-7b57d286ba23 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:45.563103 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d4e520-bd6f-404c-ba8f-7b57d286ba23 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:45.564036 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d4e520-bd6f-404c-ba8f-7b57d286ba23 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:45.566089 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d4e520-bd6f-404c-ba8f-7b57d286ba23 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:45.567967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a9165-22b6-42c4-81a4-e159c1e23f9c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:45.570734 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:45.595618 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 242/477] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 12:20:45 2022] POST /identity/v3/users => generated 249 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:20:45.627814 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f8c3a-0a7c-42c5-8d97-00b5e8651388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:45.628968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f8c3a-0a7c-42c5-8d97-00b5e8651388 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:45.629453 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f8c3a-0a7c-42c5-8d97-00b5e8651388 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:45.630111 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f8c3a-0a7c-42c5-8d97-00b5e8651388 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:45.631392 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f8c3a-0a7c-42c5-8d97-00b5e8651388 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:45.633263 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f8c3a-0a7c-42c5-8d97-00b5e8651388 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:45.645644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.sql.core [None req-db5f8c3a-0a7c-42c5-8d97-00b5e8651388 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 29 12:20:45.645644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 29 12:20:45.645644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [parameters: {'id': '891f68476aa846e68e2a4a2ccb7d76b2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 29 12:20:45.645644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80170) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 29 12:20:45.664899 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-db5f8c3a-0a7c-42c5-8d97-00b5e8651388 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 29 12:20:45.664899 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:45.664899 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 12:20:45.664899 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 12:20:45.664899 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 12:20:45.664899 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 12:20:45.664899 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 12:20:45.664899 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 12:20:45.664899 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 12:20:45.664899 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application conn.query(q) Jun 29 12:20:45.664899 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 12:20:45.664899 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 12:20:45.664899 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 12:20:45.664899 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result.read() Jun 29 12:20:45.664899 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 12:20:45.665875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 12:20:45.665875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 12:20:45.665875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 12:20:45.665875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 12:20:45.665875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 12:20:45.665875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 12:20:45.665875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 12:20:45.665875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 29 12:20:45.665875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:45.665875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 29 12:20:45.665875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:45.665875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:45.665875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 29 12:20:45.665875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 29 12:20:45.665875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 29 12:20:45.665875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return ref.to_dict() Jun 29 12:20:45.665875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 12:20:45.665875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application next(self.gen) Jun 29 12:20:45.667706 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 29 12:20:45.667706 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application yield resource Jun 29 12:20:45.667706 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 12:20:45.667706 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application next(self.gen) Jun 29 12:20:45.667706 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 29 12:20:45.667706 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.session.rollback() Jun 29 12:20:45.667706 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 29 12:20:45.667706 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.force_reraise() Jun 29 12:20:45.667706 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 29 12:20:45.667706 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise self.value Jun 29 12:20:45.667706 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 29 12:20:45.667706 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 29 12:20:45.667706 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 29 12:20:45.667706 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application session.commit() Jun 29 12:20:45.667706 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 29 12:20:45.667706 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 29 12:20:45.667706 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 29 12:20:45.669245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._prepare_impl() Jun 29 12:20:45.669245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 29 12:20:45.669245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.session.flush() Jun 29 12:20:45.669245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 29 12:20:45.669245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._flush(objects) Jun 29 12:20:45.669245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 29 12:20:45.669245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 29 12:20:45.669245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 29 12:20:45.669245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application compat.raise_( Jun 29 12:20:45.669245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 12:20:45.669245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception Jun 29 12:20:45.669245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 29 12:20:45.669245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application flush_context.execute() Jun 29 12:20:45.669245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 29 12:20:45.669245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rec.execute(self) Jun 29 12:20:45.669245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 29 12:20:45.669245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 29 12:20:45.670104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 29 12:20:45.670104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 29 12:20:45.670104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 29 12:20:45.670104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 29 12:20:45.670104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 29 12:20:45.670104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 29 12:20:45.670104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 29 12:20:45.670104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 29 12:20:45.670104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 29 12:20:45.670104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 29 12:20:45.670104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 29 12:20:45.670104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 29 12:20:45.670104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 29 12:20:45.670104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 29 12:20:45.670104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 12:20:45.670104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception Jun 29 12:20:45.670967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 12:20:45.670967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 12:20:45.670967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 12:20:45.670967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 12:20:45.670967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 12:20:45.670967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 12:20:45.670967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 12:20:45.670967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application conn.query(q) Jun 29 12:20:45.670967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 12:20:45.670967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 12:20:45.670967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 12:20:45.670967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result.read() Jun 29 12:20:45.670967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 12:20:45.670967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 12:20:45.670967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 12:20:45.670967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 12:20:45.673100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 12:20:45.673100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 12:20:45.673100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 12:20:45.673100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 12:20:45.673100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 29 12:20:45.673100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 29 12:20:45.673100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [parameters: {'id': '891f68476aa846e68e2a4a2ccb7d76b2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 29 12:20:45.673100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 29 12:20:45.673100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:45.673100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:20:45.673100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:45.673100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:45.673100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:45.673100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:45.673100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:45.673100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:45.675120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:45.675120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:45.675120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:45.675120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:45.675120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:45.675120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:45.675120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 29 12:20:45.675120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 29 12:20:45.675120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:45.675120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:45.675120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 29 12:20:45.675120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 29 12:20:45.675120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 29 12:20:45.675120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 29 12:20:45.675120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 29 12:20:45.675120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:45.678903 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 236/478] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Jun 29 12:20:45 2022] POST /identity/v3/roles => generated 144 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:20:45.685143 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4e520-bd6f-404c-ba8f-7b57d286ba23 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:45.737229 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:45.810515 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d4e520-bd6f-404c-ba8f-7b57d286ba23 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:45.847904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00027fde-ff4b-4e44-a92b-2e20ddc909b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:45.848720 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00027fde-ff4b-4e44-a92b-2e20ddc909b1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:45.849005 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00027fde-ff4b-4e44-a92b-2e20ddc909b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:45.849287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00027fde-ff4b-4e44-a92b-2e20ddc909b1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:45.861747 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00027fde-ff4b-4e44-a92b-2e20ddc909b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:45.870177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-00027fde-ff4b-4e44-a92b-2e20ddc909b1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 12:20:45.870177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:45.870177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:45.870177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:45.870177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:45.870177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:45.870177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:45.870177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:45.870177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:45.870177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:45.870177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:45.870177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:45.870177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:45.870177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:45.870177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:45.870177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:45.871107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:45.871107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:45.871107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:45.871107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:45.871107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:45.871107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:45.871107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:45.871107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:45.871107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:45.871107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:45.871107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:45.871107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:45.871107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:45.871107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:45.871107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:45.871107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:45.871900 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:45.871900 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:45.871900 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:45.871900 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:45.871900 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:45.871900 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:45.871900 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:45.871900 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:45.871900 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:45.871900 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:45.871900 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 29 12:20:45.871900 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:45.872638 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 243/479] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 12:20:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:45.877085 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f8c3a-0a7c-42c5-8d97-00b5e8651388 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:45.983259 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f8c3a-0a7c-42c5-8d97-00b5e8651388 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:46.018080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc343cda-f779-4349-ae66-bed1fd379c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:46.018785 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc343cda-f779-4349-ae66-bed1fd379c24 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:46.019072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc343cda-f779-4349-ae66-bed1fd379c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:46.019356 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc343cda-f779-4349-ae66-bed1fd379c24 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:46.020106 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc343cda-f779-4349-ae66-bed1fd379c24 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:46.020415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc343cda-f779-4349-ae66-bed1fd379c24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:46.022045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc343cda-f779-4349-ae66-bed1fd379c24 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:46.031693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 237/480] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:20:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:47.286339 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c5877-928a-409d-ad19-d1f194902916 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:47.287087 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c5877-928a-409d-ad19-d1f194902916 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:47.287087 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c5877-928a-409d-ad19-d1f194902916 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:47.288959 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 244/481] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:47.295233 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16bfb32-bac3-4d38-91f0-388414c011e4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:47.295831 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16bfb32-bac3-4d38-91f0-388414c011e4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:47.295831 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16bfb32-bac3-4d38-91f0-388414c011e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:47.319538 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d16bfb32-bac3-4d38-91f0-388414c011e4 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:47.320755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d16bfb32-bac3-4d38-91f0-388414c011e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:47.351771 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:47.412568 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 238/482] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:20:47.453023 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c5877-928a-409d-ad19-d1f194902916 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:47.472416 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0d955-5108-413b-b6f2-7d987874a701 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:47.473149 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0d955-5108-413b-b6f2-7d987874a701 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:47.473633 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0d955-5108-413b-b6f2-7d987874a701 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:47.475433 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 239/483] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:47.481655 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08336536-5459-4a61-abee-683b59f6c94f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:47.482382 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08336536-5459-4a61-abee-683b59f6c94f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:47.482897 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08336536-5459-4a61-abee-683b59f6c94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:47.502221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-08336536-5459-4a61-abee-683b59f6c94f None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:47.503988 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-08336536-5459-4a61-abee-683b59f6c94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:47.608888 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 240/484] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:20:47.635532 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08336536-5459-4a61-abee-683b59f6c94f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:47.645737 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:47.670481 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4c5877-928a-409d-ad19-d1f194902916 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:47.728377 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff857efc-58f9-4450-a7f5-8e3b7a668008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:47.729506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff857efc-58f9-4450-a7f5-8e3b7a668008 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:47.729687 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff857efc-58f9-4450-a7f5-8e3b7a668008 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:47.729928 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff857efc-58f9-4450-a7f5-8e3b7a668008 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:47.759564 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff857efc-58f9-4450-a7f5-8e3b7a668008 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:47.759970 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:47.768366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ff857efc-58f9-4450-a7f5-8e3b7a668008 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:20:47.768366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:47.768366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:47.768366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:47.768366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:47.768366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:47.768366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:47.768366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:47.768366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:47.768366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:47.768366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:47.768366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:47.768366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:47.768366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:47.768366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:47.768366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:47.769344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:47.769344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:47.769344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:47.769344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:47.769344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:47.769344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:47.769344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:47.769344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:47.769344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:47.769344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:47.769344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:47.769344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:47.769344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:47.769344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:47.769344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:47.769344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:47.770133 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:47.770133 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:47.770133 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:47.770133 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:47.770133 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:47.770133 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:47.770133 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:47.770133 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:47.770133 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:47.770133 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:47.770133 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:20:47.770133 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:47.773513 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08336536-5459-4a61-abee-683b59f6c94f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:47.778915 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 245/485] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 12:20:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:47.787900 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff857efc-58f9-4450-a7f5-8e3b7a668008 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:47.815477 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef9edba-1516-4baa-b822-f4015a034f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:47.816281 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef9edba-1516-4baa-b822-f4015a034f6d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:47.816567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef9edba-1516-4baa-b822-f4015a034f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:47.816844 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef9edba-1516-4baa-b822-f4015a034f6d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:47.827415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef9edba-1516-4baa-b822-f4015a034f6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:47.844071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0ef9edba-1516-4baa-b822-f4015a034f6d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:20:47.844071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:47.844071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:47.844071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:47.844071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:47.844071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:47.844071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:47.844071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:47.844071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:47.844071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:47.844071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:47.844071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:47.844071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:47.844071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:47.844071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:47.844071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:47.844998 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:47.844998 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:47.844998 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:47.844998 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:47.844998 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:47.844998 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:47.844998 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:47.844998 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:47.844998 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:47.844998 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:47.844998 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:47.844998 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:47.844998 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:47.844998 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:47.844998 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:47.844998 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:47.845791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:47.845791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:47.845791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:47.845791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:47.845791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:47.845791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:47.845791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:47.845791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:47.845791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:47.845791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:47.845791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:20:47.845791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:47.845791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 241/486] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 12:20:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:20:47.848389 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa9b8b-e53d-49dc-a1a6-283e56e0ee0b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:47.848680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa9b8b-e53d-49dc-a1a6-283e56e0ee0b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:47.848958 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa9b8b-e53d-49dc-a1a6-283e56e0ee0b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:47.854229 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 242/487] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:47.855072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa9b8b-e53d-49dc-a1a6-283e56e0ee0b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:47.898535 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff857efc-58f9-4450-a7f5-8e3b7a668008 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:47.933946 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f11ef4-2f56-466e-89b9-3ca841a65643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:47.934639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f11ef4-2f56-466e-89b9-3ca841a65643 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:47.934928 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f11ef4-2f56-466e-89b9-3ca841a65643 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:47.935259 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f11ef4-2f56-466e-89b9-3ca841a65643 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:47.936010 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f11ef4-2f56-466e-89b9-3ca841a65643 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:47.936364 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f11ef4-2f56-466e-89b9-3ca841a65643 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:47.937749 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f11ef4-2f56-466e-89b9-3ca841a65643 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:47.946536 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 246/488] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 12:20:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:47.948299 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb964577-f016-4799-ba87-26749113cc50 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:47.948768 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb964577-f016-4799-ba87-26749113cc50 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:47.949048 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb964577-f016-4799-ba87-26749113cc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:47.967268 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-eb964577-f016-4799-ba87-26749113cc50 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:47.968559 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-eb964577-f016-4799-ba87-26749113cc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:47.979803 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa9b8b-e53d-49dc-a1a6-283e56e0ee0b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:47.980489 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:48.015260 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98040f5e-5618-402a-82e6-712932e26584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:48.015967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98040f5e-5618-402a-82e6-712932e26584 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:48.016500 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98040f5e-5618-402a-82e6-712932e26584 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:48.016651 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98040f5e-5618-402a-82e6-712932e26584 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:48.017415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98040f5e-5618-402a-82e6-712932e26584 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:48.017728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98040f5e-5618-402a-82e6-712932e26584 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:48.019268 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98040f5e-5618-402a-82e6-712932e26584 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:48.028740 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 243/489] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 12:20:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:48.029682 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98040f5e-5618-402a-82e6-712932e26584 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:48.041870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:48.043541 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 247/490] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:48.044525 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb964577-f016-4799-ba87-26749113cc50 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:48.164627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98040f5e-5618-402a-82e6-712932e26584 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:48.168965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb964577-f016-4799-ba87-26749113cc50 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:48.206327 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff189e3c-dd0f-4f2b-affb-d6ba8b849bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:48.207175 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff189e3c-dd0f-4f2b-affb-d6ba8b849bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:48.207628 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff189e3c-dd0f-4f2b-affb-d6ba8b849bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:48.207732 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff189e3c-dd0f-4f2b-affb-d6ba8b849bd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:48.213107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d22f4-b269-45cf-89b0-4a6977bb8206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:48.214286 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d22f4-b269-45cf-89b0-4a6977bb8206 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:48.214819 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d22f4-b269-45cf-89b0-4a6977bb8206 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:48.215826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d22f4-b269-45cf-89b0-4a6977bb8206 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:48.219875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff189e3c-dd0f-4f2b-affb-d6ba8b849bd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:48.221324 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff189e3c-dd0f-4f2b-affb-d6ba8b849bd8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:48.229546 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122d22f4-b269-45cf-89b0-4a6977bb8206 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:48.231902 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122d22f4-b269-45cf-89b0-4a6977bb8206 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:48.239324 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:48.240541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ff189e3c-dd0f-4f2b-affb-d6ba8b849bd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:48.240541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:48.240541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:48.240541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:48.240541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:48.240541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:48.240541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:48.240541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:48.240541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:48.240541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:48.240541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:48.240541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:48.240541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:20:48.240541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:20:48.240541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:20:48.241383 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:20:48.241383 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:48.241383 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:48.241383 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:48.241383 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:48.241383 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:48.241383 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:48.241383 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:48.241383 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:48.241383 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:48.241383 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:48.241383 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:48.241383 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:48.241383 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:48.241383 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:48.241383 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:48.241383 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:48.242233 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:20:48.242233 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:20:48.242233 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:48.242233 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:48.246391 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 244/491] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:48.247299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff189e3c-dd0f-4f2b-affb-d6ba8b849bd8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:48.255856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-122d22f4-b269-45cf-89b0-4a6977bb8206 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:48.255856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:48.255856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:48.255856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:48.255856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:48.255856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:48.255856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:48.255856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:48.255856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:48.255856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:48.255856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:48.255856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:48.255856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:20:48.255856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:20:48.255856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:20:48.256717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:20:48.256717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:48.256717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:48.256717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:48.256717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:48.256717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:48.256717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:48.256717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:48.256717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:48.256717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:48.256717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:48.256717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:48.256717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:48.256717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:48.256717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:48.256717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:48.256717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:48.257541 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:20:48.257541 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:20:48.257541 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:48.257541 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:48.259963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 248/492] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:48.262071 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d22f4-b269-45cf-89b0-4a6977bb8206 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:48.326736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:48.375819 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d22f4-b269-45cf-89b0-4a6977bb8206 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:48.382623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff189e3c-dd0f-4f2b-affb-d6ba8b849bd8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:48.418947 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7169005-e357-4753-a18e-f708423958d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:48.419836 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7169005-e357-4753-a18e-f708423958d5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:48.420262 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7169005-e357-4753-a18e-f708423958d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:48.420497 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcde5b5-df32-4d45-b64c-c3272fdbc047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:48.420873 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7169005-e357-4753-a18e-f708423958d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:48.421499 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcde5b5-df32-4d45-b64c-c3272fdbc047 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:48.421882 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcde5b5-df32-4d45-b64c-c3272fdbc047 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:48.422122 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7169005-e357-4753-a18e-f708423958d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:48.422337 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcde5b5-df32-4d45-b64c-c3272fdbc047 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:48.422749 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7169005-e357-4753-a18e-f708423958d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:48.424271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7169005-e357-4753-a18e-f708423958d5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:48.434891 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcde5b5-df32-4d45-b64c-c3272fdbc047 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:48.439318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 249/493] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:48.440348 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7169005-e357-4753-a18e-f708423958d5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:48.447859 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9bcde5b5-df32-4d45-b64c-c3272fdbc047 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:20:48.447859 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:48.447859 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:48.447859 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:48.447859 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:48.447859 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:48.447859 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:48.447859 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:48.447859 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:48.447859 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:48.447859 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:48.447859 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:48.447859 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:48.447859 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:48.447859 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:48.447859 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:48.449008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:48.449008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:48.449008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:48.449008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:48.449008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:48.449008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:48.449008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:48.449008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:48.449008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:48.449008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:48.449008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:48.449008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:48.449008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:48.449008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:48.449008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:48.449008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:48.449841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:48.449841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:48.449841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:48.449841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:48.449841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:48.449841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:48.449841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:48.449841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:48.449841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:48.449841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:48.449841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:20:48.449841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:48.454497 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 245/494] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 12:20:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:48.456662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcde5b5-df32-4d45-b64c-c3272fdbc047 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:48.506311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:48.550911 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7169005-e357-4753-a18e-f708423958d5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:48.567918 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcde5b5-df32-4d45-b64c-c3272fdbc047 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:48.585696 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:48.586054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4e40c-70e4-4f25-a4d6-3a28d8d69826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:48.586780 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4e40c-70e4-4f25-a4d6-3a28d8d69826 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:48.587150 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4e40c-70e4-4f25-a4d6-3a28d8d69826 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:48.587282 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4e40c-70e4-4f25-a4d6-3a28d8d69826 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:48.588077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c4e40c-70e4-4f25-a4d6-3a28d8d69826 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:48.589068 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c4e40c-70e4-4f25-a4d6-3a28d8d69826 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:48.590659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c4e40c-70e4-4f25-a4d6-3a28d8d69826 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:48.609206 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 250/495] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:48.609206 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca54de8-817c-4473-9537-7650980ce6bf None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:48.609206 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca54de8-817c-4473-9537-7650980ce6bf None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:48.609206 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca54de8-817c-4473-9537-7650980ce6bf None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:48.609206 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 251/496] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:48.609206 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca54de8-817c-4473-9537-7650980ce6bf None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:48.613047 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059fd02-4489-41fa-bb96-2805aaacc2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:48.613871 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059fd02-4489-41fa-bb96-2805aaacc2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:48.614161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059fd02-4489-41fa-bb96-2805aaacc2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:48.614469 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0059fd02-4489-41fa-bb96-2805aaacc2d0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:48.622609 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059fd02-4489-41fa-bb96-2805aaacc2d0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:48.624128 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0059fd02-4489-41fa-bb96-2805aaacc2d0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:48.640717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0059fd02-4489-41fa-bb96-2805aaacc2d0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 29 12:20:48.640717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:48.640717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:48.640717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:48.640717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:48.640717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:48.640717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:48.640717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:48.640717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:48.640717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:48.640717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:48.640717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:48.640717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:20:48.640717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:20:48.640717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:20:48.640717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:20:48.641662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:48.641662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:48.641662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:20:48.641662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:20:48.641662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:20:48.641662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:20:48.641662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:48.641662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:48.641662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:48.641662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:48.641662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:48.641662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:48.641662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:48.641662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:48.641662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:48.641662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:48.641662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:48.641662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:48.643055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:48.643055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:48.643055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:20:48.643055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:20:48.643055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:20:48.643055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:20:48.643055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:20:48.643055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:20:48.643055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 29 12:20:48.643055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:48.643055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 246/497] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:48.643055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059fd02-4489-41fa-bb96-2805aaacc2d0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:48.728086 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca54de8-817c-4473-9537-7650980ce6bf None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:48.747013 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:48.748703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0059fd02-4489-41fa-bb96-2805aaacc2d0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:48.765036 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ff7c7e-0ed2-491f-95e3-59d0aa89307c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:48.765700 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff7c7e-0ed2-491f-95e3-59d0aa89307c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:48.765983 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff7c7e-0ed2-491f-95e3-59d0aa89307c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:48.766259 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ff7c7e-0ed2-491f-95e3-59d0aa89307c None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:48.767027 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff7c7e-0ed2-491f-95e3-59d0aa89307c None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:48.767333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff7c7e-0ed2-491f-95e3-59d0aa89307c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:48.768737 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ff7c7e-0ed2-491f-95e3-59d0aa89307c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:48.777068 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 252/498] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 12:20:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:48.778262 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4246e8-6328-46ca-bdfb-fef0c2a7be5c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:48.778582 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4246e8-6328-46ca-bdfb-fef0c2a7be5c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:48.778863 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4246e8-6328-46ca-bdfb-fef0c2a7be5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:48.789699 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6fe5ab-cc66-438e-81d9-5d46dec5b5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:48.790721 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fe5ab-cc66-438e-81d9-5d46dec5b5ea None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:48.791012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fe5ab-cc66-438e-81d9-5d46dec5b5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:48.791843 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fe5ab-cc66-438e-81d9-5d46dec5b5ea None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:48.796619 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bf4246e8-6328-46ca-bdfb-fef0c2a7be5c None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:48.797798 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bf4246e8-6328-46ca-bdfb-fef0c2a7be5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:48.800422 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6fe5ab-cc66-438e-81d9-5d46dec5b5ea None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:48.801912 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6fe5ab-cc66-438e-81d9-5d46dec5b5ea None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:48.816708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5e6fe5ab-cc66-438e-81d9-5d46dec5b5ea None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 12:20:48.816708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:48.816708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:48.816708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:48.816708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:48.816708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:48.816708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:48.816708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:48.816708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:48.816708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:48.816708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:48.816708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:48.816708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:20:48.816708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:20:48.816708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:20:48.816708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:20:48.817798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:48.817798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:48.817798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:20:48.817798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:20:48.817798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:20:48.817798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:20:48.817798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:48.817798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:48.817798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:48.817798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:48.817798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:48.817798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:48.817798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:48.817798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:48.817798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:48.817798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:48.817798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:48.817798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:48.819164 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:48.819164 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:48.819164 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:20:48.819164 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:20:48.819164 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:20:48.819164 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:20:48.819164 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:20:48.819164 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:20:48.819164 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 12:20:48.819164 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:48.819164 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 247/499] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Jun 29 12:20:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:48.819164 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6fe5ab-cc66-438e-81d9-5d46dec5b5ea None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:48.858656 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:48.869063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 253/500] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:48.870542 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803ad9d-d9fe-4332-8ff4-bcda835fdbb2 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:48.870895 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803ad9d-d9fe-4332-8ff4-bcda835fdbb2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:48.871246 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1803ad9d-d9fe-4332-8ff4-bcda835fdbb2 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:48.874458 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 254/501] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:48.874458 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803ad9d-d9fe-4332-8ff4-bcda835fdbb2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:48.923128 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6fe5ab-cc66-438e-81d9-5d46dec5b5ea None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:48.956658 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02877d5a-8274-49df-86a3-8307432d1356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:48.957325 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02877d5a-8274-49df-86a3-8307432d1356 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:48.957619 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02877d5a-8274-49df-86a3-8307432d1356 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:48.957904 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02877d5a-8274-49df-86a3-8307432d1356 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:48.958778 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02877d5a-8274-49df-86a3-8307432d1356 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:48.959096 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02877d5a-8274-49df-86a3-8307432d1356 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:48.960535 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02877d5a-8274-49df-86a3-8307432d1356 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:48.980747 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1803ad9d-d9fe-4332-8ff4-bcda835fdbb2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:48.990924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:48.997508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 248/502] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Jun 29 12:20:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:48.998356 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02877d5a-8274-49df-86a3-8307432d1356 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:49.015581 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67cd67-d06b-4c15-91a7-94ac94254315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:49.016437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67cd67-d06b-4c15-91a7-94ac94254315 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:49.016785 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67cd67-d06b-4c15-91a7-94ac94254315 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:49.017067 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67cd67-d06b-4c15-91a7-94ac94254315 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:49.027598 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f67cd67-d06b-4c15-91a7-94ac94254315 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:49.029043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f67cd67-d06b-4c15-91a7-94ac94254315 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:49.046106 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9f67cd67-d06b-4c15-91a7-94ac94254315 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:49.046106 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:49.046106 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:49.046106 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:49.046106 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:49.046106 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:49.046106 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:49.046106 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:49.046106 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:49.046106 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:49.046106 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:49.046106 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:49.046106 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:20:49.046106 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:20:49.046106 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:20:49.047224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:20:49.047224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:49.047224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:49.047224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:49.047224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:49.047224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:49.047224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:49.047224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:49.047224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:49.047224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:49.047224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:49.047224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:49.047224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:49.047224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:49.047224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:49.047224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:49.047224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:49.052852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:20:49.052852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:20:49.052852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:49.052852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:49.052852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 255/503] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:49.052852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad554e-1a84-4d28-a58a-457555d075e6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:49.052852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad554e-1a84-4d28-a58a-457555d075e6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:49.052852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad554e-1a84-4d28-a58a-457555d075e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:49.065243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a8ad554e-1a84-4d28-a58a-457555d075e6 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:49.066408 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a8ad554e-1a84-4d28-a58a-457555d075e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:49.090735 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:49.097402 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02877d5a-8274-49df-86a3-8307432d1356 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:49.133397 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 256/504] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:49.133587 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5d7b9-4fd3-4eb7-8c88-5f195e97fa69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:49.133693 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad554e-1a84-4d28-a58a-457555d075e6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:49.133765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5d7b9-4fd3-4eb7-8c88-5f195e97fa69 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:49.133765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5d7b9-4fd3-4eb7-8c88-5f195e97fa69 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:49.133765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5d7b9-4fd3-4eb7-8c88-5f195e97fa69 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:49.133765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5d7b9-4fd3-4eb7-8c88-5f195e97fa69 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:49.133765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5d7b9-4fd3-4eb7-8c88-5f195e97fa69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:49.134698 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5d7b9-4fd3-4eb7-8c88-5f195e97fa69 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:49.160401 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 249/505] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Jun 29 12:20:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:49.161239 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5d7b9-4fd3-4eb7-8c88-5f195e97fa69 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:49.222459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:49.231829 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad554e-1a84-4d28-a58a-457555d075e6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:49.261133 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5d7b9-4fd3-4eb7-8c88-5f195e97fa69 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:49.265245 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ee7321-975e-402c-b3cd-7a3cf98c7922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:49.266034 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee7321-975e-402c-b3cd-7a3cf98c7922 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:49.266315 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee7321-975e-402c-b3cd-7a3cf98c7922 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:49.266624 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee7321-975e-402c-b3cd-7a3cf98c7922 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:49.277025 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ee7321-975e-402c-b3cd-7a3cf98c7922 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:49.284886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b1ee7321-975e-402c-b3cd-7a3cf98c7922 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:20:49.284886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:49.284886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:49.284886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:49.284886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:49.284886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:49.284886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:49.284886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:49.284886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:49.284886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:49.284886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:49.284886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:49.284886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:49.284886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:49.284886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:49.284886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:49.285987 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:49.285987 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:49.285987 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:49.285987 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:49.285987 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:49.285987 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:49.285987 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:49.285987 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:49.285987 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:49.285987 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:49.285987 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:49.285987 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:49.285987 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:49.285987 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:49.285987 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:49.285987 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:49.286980 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:49.286980 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:49.286980 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:49.286980 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:49.286980 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:49.286980 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:49.286980 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:49.286980 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:49.286980 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:49.286980 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:49.286980 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:20:49.286980 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:49.291107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 257/506] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 12:20:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:49.291928 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ee7321-975e-402c-b3cd-7a3cf98c7922 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:49.295313 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520af18b-da11-415c-81df-eed78b1a2d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:49.295955 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520af18b-da11-415c-81df-eed78b1a2d42 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:49.296238 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520af18b-da11-415c-81df-eed78b1a2d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:49.296512 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520af18b-da11-415c-81df-eed78b1a2d42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:49.320694 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:49.366508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520af18b-da11-415c-81df-eed78b1a2d42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:49.367993 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520af18b-da11-415c-81df-eed78b1a2d42 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:49.397131 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ee7321-975e-402c-b3cd-7a3cf98c7922 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:49.430379 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bcb158-1471-4e65-aed0-bdfac6ee666b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:49.431056 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bcb158-1471-4e65-aed0-bdfac6ee666b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:49.431340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bcb158-1471-4e65-aed0-bdfac6ee666b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:49.431619 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bcb158-1471-4e65-aed0-bdfac6ee666b None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:49.432426 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bcb158-1471-4e65-aed0-bdfac6ee666b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:49.432811 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bcb158-1471-4e65-aed0-bdfac6ee666b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:49.434176 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bcb158-1471-4e65-aed0-bdfac6ee666b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:49.445505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 258/507] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:20:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:49.446301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bcb158-1471-4e65-aed0-bdfac6ee666b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:49.460555 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:49.472620 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 250/508] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:20:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:20:49.473472 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520af18b-da11-415c-81df-eed78b1a2d42 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:49.527245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:49.547097 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bcb158-1471-4e65-aed0-bdfac6ee666b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:49.571259 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520af18b-da11-415c-81df-eed78b1a2d42 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:49.580109 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083c5be-07e9-4386-a553-616fe3ee139b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:49.580895 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083c5be-07e9-4386-a553-616fe3ee139b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:49.581175 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083c5be-07e9-4386-a553-616fe3ee139b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:49.581449 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083c5be-07e9-4386-a553-616fe3ee139b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:49.591839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8083c5be-07e9-4386-a553-616fe3ee139b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:49.593326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8083c5be-07e9-4386-a553-616fe3ee139b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:49.604716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480cab8a-6c93-43ee-88b2-777d80c38ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:49.605354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480cab8a-6c93-43ee-88b2-777d80c38ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:49.605635 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480cab8a-6c93-43ee-88b2-777d80c38ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:49.605908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480cab8a-6c93-43ee-88b2-777d80c38ea1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:49.610438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-8083c5be-07e9-4386-a553-616fe3ee139b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:49.610438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:49.610438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:49.610438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:49.610438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:49.610438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:49.610438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:49.610438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:49.610438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:49.610438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:49.610438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:49.610438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:49.610438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:20:49.610438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:20:49.610438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:20:49.611432 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:20:49.611432 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:49.611432 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:49.611432 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:49.611432 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:49.611432 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:49.611432 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:49.611432 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:49.611432 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:49.611432 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:49.611432 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:49.611432 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:49.611432 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:49.611432 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:49.611432 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:49.611432 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:49.611432 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:49.612487 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:20:49.612487 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:20:49.612487 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:49.612487 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:49.612487 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 259/509] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:49.612487 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083c5be-07e9-4386-a553-616fe3ee139b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:49.672685 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480cab8a-6c93-43ee-88b2-777d80c38ea1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:49.674335 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480cab8a-6c93-43ee-88b2-777d80c38ea1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:49.677097 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:49.714275 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083c5be-07e9-4386-a553-616fe3ee139b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:49.727196 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:49.747948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b00d7-991c-45ea-b2e6-d89f353be152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:49.748617 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b00d7-991c-45ea-b2e6-d89f353be152 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:49.748906 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b00d7-991c-45ea-b2e6-d89f353be152 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:49.749195 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b00d7-991c-45ea-b2e6-d89f353be152 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:49.749941 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b00d7-991c-45ea-b2e6-d89f353be152 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:49.750274 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b00d7-991c-45ea-b2e6-d89f353be152 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:49.751701 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b00d7-991c-45ea-b2e6-d89f353be152 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:49.759478 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 260/510] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 12:20:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:49.760783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b00d7-991c-45ea-b2e6-d89f353be152 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:49.771650 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 251/511] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:20:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:20:49.772470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480cab8a-6c93-43ee-88b2-777d80c38ea1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:49.860715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b00d7-991c-45ea-b2e6-d89f353be152 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:49.879332 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480cab8a-6c93-43ee-88b2-777d80c38ea1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:49.884584 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:49.893156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a62cb1-b4c2-43df-b828-e077a14581fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:49.893939 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a62cb1-b4c2-43df-b828-e077a14581fd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:49.894221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a62cb1-b4c2-43df-b828-e077a14581fd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:49.894527 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a62cb1-b4c2-43df-b828-e077a14581fd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:49.902818 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a62cb1-b4c2-43df-b828-e077a14581fd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:49.904256 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a62cb1-b4c2-43df-b828-e077a14581fd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:49.915311 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776508d-e71d-415d-8213-25a7991b360c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:49.916099 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776508d-e71d-415d-8213-25a7991b360c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:49.916947 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776508d-e71d-415d-8213-25a7991b360c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:49.917208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9776508d-e71d-415d-8213-25a7991b360c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:49.919717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-76a62cb1-b4c2-43df-b828-e077a14581fd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 12:20:49.919717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:49.919717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:49.919717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:49.919717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:49.919717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:49.919717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:49.919717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:49.919717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:49.919717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:49.919717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:49.919717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:49.919717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:20:49.919717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:20:49.919717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:20:49.919717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:20:49.920692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:49.920692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:49.920692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:20:49.920692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:20:49.920692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:20:49.920692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:20:49.920692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:49.920692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:49.920692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:49.920692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:49.920692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:49.920692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:49.920692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:49.920692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:49.920692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:49.920692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:49.920692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:49.920692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:49.921605 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:49.921605 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:49.921605 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:20:49.921605 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:20:49.921605 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:20:49.921605 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:20:49.921605 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:20:49.921605 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:20:49.921605 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 12:20:49.921605 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:49.925228 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 261/512] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:49.926255 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a62cb1-b4c2-43df-b828-e077a14581fd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:49.927503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9776508d-e71d-415d-8213-25a7991b360c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:49.928950 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9776508d-e71d-415d-8213-25a7991b360c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:49.947170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9776508d-e71d-415d-8213-25a7991b360c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:49.947170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:49.947170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:49.947170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:49.947170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:49.947170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:49.947170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:49.947170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:49.947170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:49.947170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:49.947170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:49.947170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:49.947170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:20:49.947170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:20:49.947170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:20:49.948561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:20:49.948561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:49.948561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:49.948561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:49.948561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:49.948561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:49.948561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:49.948561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:49.948561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:49.948561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:49.948561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:49.948561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:49.948561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:49.948561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:49.948561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:49.948561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:49.948561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:49.949398 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:20:49.949398 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:20:49.949398 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:49.949398 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:49.949398 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 252/513] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:49.949398 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776508d-e71d-415d-8213-25a7991b360c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:49.958684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:50.027467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a62cb1-b4c2-43df-b828-e077a14581fd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:50.050063 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9776508d-e71d-415d-8213-25a7991b360c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:50.059949 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04351eb3-9d2e-491a-a886-e3bb5ddba54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:50.060596 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04351eb3-9d2e-491a-a886-e3bb5ddba54a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:50.060884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04351eb3-9d2e-491a-a886-e3bb5ddba54a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:50.061161 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04351eb3-9d2e-491a-a886-e3bb5ddba54a None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:50.061874 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04351eb3-9d2e-491a-a886-e3bb5ddba54a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:50.062180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04351eb3-9d2e-491a-a886-e3bb5ddba54a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:50.063595 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04351eb3-9d2e-491a-a886-e3bb5ddba54a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:50.075223 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 262/514] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:20:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:50.075956 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04351eb3-9d2e-491a-a886-e3bb5ddba54a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:50.088708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa1d28-916c-4c26-b13f-c2884239e6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:50.089506 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa1d28-916c-4c26-b13f-c2884239e6cc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:50.089798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa1d28-916c-4c26-b13f-c2884239e6cc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:50.090083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa1d28-916c-4c26-b13f-c2884239e6cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:50.100645 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa1d28-916c-4c26-b13f-c2884239e6cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:50.102109 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa1d28-916c-4c26-b13f-c2884239e6cc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:50.121176 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:50.121395 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f7fa1d28-916c-4c26-b13f-c2884239e6cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:50.121395 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:50.121395 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:50.121395 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:50.121395 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:50.121395 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:50.121395 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:50.121395 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:50.121395 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:50.121395 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:50.121395 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:50.121395 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:50.121395 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:20:50.121395 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:20:50.121395 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:20:50.122223 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:20:50.122223 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:50.122223 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:50.122223 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:50.122223 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:50.122223 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:50.122223 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:50.122223 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:50.122223 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:50.122223 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:50.122223 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:50.122223 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:50.122223 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:50.122223 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:50.122223 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:50.122223 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:50.122223 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:50.123414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:20:50.123414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:20:50.123414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:50.123414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:50.123414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 253/515] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:50.123414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa1d28-916c-4c26-b13f-c2884239e6cc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:50.178665 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04351eb3-9d2e-491a-a886-e3bb5ddba54a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:50.183224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:50.212678 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258525cc-17e1-4db6-81d9-080ed9e0910e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:50.213682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258525cc-17e1-4db6-81d9-080ed9e0910e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:50.213963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258525cc-17e1-4db6-81d9-080ed9e0910e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:50.214231 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258525cc-17e1-4db6-81d9-080ed9e0910e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:50.224078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258525cc-17e1-4db6-81d9-080ed9e0910e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:50.224452 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa1d28-916c-4c26-b13f-c2884239e6cc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:50.225547 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258525cc-17e1-4db6-81d9-080ed9e0910e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:50.243484 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-258525cc-17e1-4db6-81d9-080ed9e0910e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:50.243484 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:50.243484 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:50.243484 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:50.243484 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:50.243484 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:50.243484 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:50.243484 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:50.243484 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:50.243484 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:50.243484 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:50.243484 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:50.243484 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:20:50.243484 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:20:50.243484 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:20:50.245147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:20:50.245147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:50.245147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:50.245147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:50.245147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:50.245147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:50.245147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:50.245147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:50.245147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:50.245147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:50.245147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:50.245147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:50.245147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:50.245147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:50.245147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:50.245147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:50.245147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:50.246176 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:20:50.246176 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:20:50.246176 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:50.246176 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:50.246176 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 263/516] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:50.246176 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258525cc-17e1-4db6-81d9-080ed9e0910e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:50.262293 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443bdd0-e596-4230-8422-8caed15646c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:50.262996 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443bdd0-e596-4230-8422-8caed15646c6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:50.263291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443bdd0-e596-4230-8422-8caed15646c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:50.263572 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443bdd0-e596-4230-8422-8caed15646c6 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:50.264420 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2443bdd0-e596-4230-8422-8caed15646c6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:50.264730 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2443bdd0-e596-4230-8422-8caed15646c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:50.266207 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2443bdd0-e596-4230-8422-8caed15646c6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:50.283463 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 254/517] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Jun 29 12:20:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:50.284249 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443bdd0-e596-4230-8422-8caed15646c6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:50.341140 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:50.345050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258525cc-17e1-4db6-81d9-080ed9e0910e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:50.380103 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf228e-c4ea-494e-9506-96f88fd7c802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:50.380712 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf228e-c4ea-494e-9506-96f88fd7c802 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:50.380997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf228e-c4ea-494e-9506-96f88fd7c802 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:50.381277 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf228e-c4ea-494e-9506-96f88fd7c802 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:50.382003 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf228e-c4ea-494e-9506-96f88fd7c802 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:50.382308 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443bdd0-e596-4230-8422-8caed15646c6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:50.382733 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf228e-c4ea-494e-9506-96f88fd7c802 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:50.383773 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bf228e-c4ea-494e-9506-96f88fd7c802 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:50.395504 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 264/518] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:50.396569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf228e-c4ea-494e-9506-96f88fd7c802 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:50.403721 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:50.422507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bb4e3f-e0d9-48af-a034-8f7a59b496d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:50.423189 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb4e3f-e0d9-48af-a034-8f7a59b496d5 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:50.423476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb4e3f-e0d9-48af-a034-8f7a59b496d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:50.423754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb4e3f-e0d9-48af-a034-8f7a59b496d5 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:50.424464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bb4e3f-e0d9-48af-a034-8f7a59b496d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:50.425895 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bb4e3f-e0d9-48af-a034-8f7a59b496d5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:50.457968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 255/519] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 12:20:50 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:20:50.457968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bb4e3f-e0d9-48af-a034-8f7a59b496d5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:50.503590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf228e-c4ea-494e-9506-96f88fd7c802 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:50.545020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4363b0e-422f-43a4-ba43-92471437175a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:50.545831 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4363b0e-422f-43a4-ba43-92471437175a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:50.546119 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4363b0e-422f-43a4-ba43-92471437175a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:50.546401 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4363b0e-422f-43a4-ba43-92471437175a None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:50.547203 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4363b0e-422f-43a4-ba43-92471437175a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:50.547564 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4363b0e-422f-43a4-ba43-92471437175a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:50.549014 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4363b0e-422f-43a4-ba43-92471437175a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:50.570484 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 265/520] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:50.570484 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4363b0e-422f-43a4-ba43-92471437175a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:50.571095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bb4e3f-e0d9-48af-a034-8f7a59b496d5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:50.596044 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:50.614570 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7acdd-9fa4-465e-8fab-0a15a014bd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:50.615541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7acdd-9fa4-465e-8fab-0a15a014bd0c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:50.615541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7acdd-9fa4-465e-8fab-0a15a014bd0c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:50.615797 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b7acdd-9fa4-465e-8fab-0a15a014bd0c None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:50.616529 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b7acdd-9fa4-465e-8fab-0a15a014bd0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:50.617035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b7acdd-9fa4-465e-8fab-0a15a014bd0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:50.619258 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b7acdd-9fa4-465e-8fab-0a15a014bd0c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:50.632635 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 256/521] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:50.633446 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7acdd-9fa4-465e-8fab-0a15a014bd0c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:50.674199 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:50.702718 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4363b0e-422f-43a4-ba43-92471437175a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:50.738668 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca45d30-d5a8-45bf-85d4-0a93e9d2290c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:50.739482 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca45d30-d5a8-45bf-85d4-0a93e9d2290c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:50.739871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca45d30-d5a8-45bf-85d4-0a93e9d2290c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:50.740239 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca45d30-d5a8-45bf-85d4-0a93e9d2290c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:50.753157 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b7acdd-9fa4-465e-8fab-0a15a014bd0c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:50.787800 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d0674-c1c7-4464-922f-6a784be96291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:50.788597 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d0674-c1c7-4464-922f-6a784be96291 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:50.788892 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d0674-c1c7-4464-922f-6a784be96291 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:50.789150 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d0674-c1c7-4464-922f-6a784be96291 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:50.797589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d0674-c1c7-4464-922f-6a784be96291 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:50.799071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2d0674-c1c7-4464-922f-6a784be96291 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:50.813519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7b2d0674-c1c7-4464-922f-6a784be96291 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:20:50.813519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:50.813519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:50.813519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:50.813519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:50.813519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:50.813519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:50.813519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:50.813519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:50.813519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:50.813519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:50.813519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:50.813519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:20:50.813519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:20:50.813519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:20:50.814449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:20:50.814449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:50.814449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:50.814449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:50.814449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:50.814449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:50.814449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:50.814449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:50.814449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:50.814449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:50.814449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:50.814449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:50.814449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:50.814449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:50.814449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:50.814449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:50.814449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:50.815319 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:20:50.815319 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:20:50.815319 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:20:50.815319 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:20:50.815319 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:20:50.815319 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:20:50.815319 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:20:50.815319 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:50.815319 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 257/522] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:20:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:50.816840 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d0674-c1c7-4464-922f-6a784be96291 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:50.821123 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca45d30-d5a8-45bf-85d4-0a93e9d2290c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:50.822751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca45d30-d5a8-45bf-85d4-0a93e9d2290c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:50.841123 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:50.918028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:50.927530 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2d0674-c1c7-4464-922f-6a784be96291 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:50.935965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 266/523] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:20:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:20:50.936884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca45d30-d5a8-45bf-85d4-0a93e9d2290c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:50.966123 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7a576-0572-42e8-a766-0553c550a94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:50.967800 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7a576-0572-42e8-a766-0553c550a94e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:50.968227 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7a576-0572-42e8-a766-0553c550a94e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:50.968796 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7a576-0572-42e8-a766-0553c550a94e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:50.979610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e7a576-0572-42e8-a766-0553c550a94e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:50.981657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e7a576-0572-42e8-a766-0553c550a94e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:51.000787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-22e7a576-0572-42e8-a766-0553c550a94e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 12:20:51.000787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:51.000787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:51.000787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:51.000787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:51.000787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:51.000787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:51.000787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:51.000787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:51.000787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:51.000787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:51.000787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:51.000787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:20:51.000787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:20:51.000787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:20:51.000787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:20:51.001868 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:51.001868 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:51.001868 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:20:51.001868 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:20:51.001868 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:20:51.001868 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:20:51.001868 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:51.001868 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:51.001868 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:51.001868 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:51.001868 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:51.001868 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:51.001868 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:51.001868 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:51.001868 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:51.001868 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:51.001868 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:51.001868 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:51.003127 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:51.003127 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:51.003127 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:20:51.003127 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:20:51.003127 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:20:51.003127 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:20:51.003127 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:20:51.003127 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:20:51.003127 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 12:20:51.003127 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:51.005329 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 258/524] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 12:20:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:51.006356 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7a576-0572-42e8-a766-0553c550a94e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:51.055071 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca45d30-d5a8-45bf-85d4-0a93e9d2290c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:51.074615 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:51.092132 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3693e-9871-4f74-84c2-fa6ad517e593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:51.092922 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3693e-9871-4f74-84c2-fa6ad517e593 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:51.093203 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3693e-9871-4f74-84c2-fa6ad517e593 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:51.093556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3693e-9871-4f74-84c2-fa6ad517e593 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:51.100582 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3693e-9871-4f74-84c2-fa6ad517e593 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:51.102021 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3693e-9871-4f74-84c2-fa6ad517e593 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:51.117706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-29e3693e-9871-4f74-84c2-fa6ad517e593 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:20:51.117706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:51.117706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:51.117706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:51.117706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:51.117706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:51.117706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:51.117706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:51.117706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:51.117706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:51.117706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:51.117706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:51.117706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:20:51.117706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:20:51.117706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:20:51.118884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:20:51.118884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:51.118884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:51.118884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:51.118884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:51.118884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:51.118884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:51.118884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:51.118884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:51.118884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:51.118884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:51.118884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:51.118884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:51.118884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:51.118884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:51.118884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:51.118884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:51.120726 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:20:51.120726 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:20:51.120726 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:20:51.120726 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:20:51.120726 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:20:51.120726 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:20:51.120726 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:20:51.120726 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:51.121327 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7a576-0572-42e8-a766-0553c550a94e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:51.124441 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 267/525] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:20:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:51.124957 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3693e-9871-4f74-84c2-fa6ad517e593 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:51.157918 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30109f8d-d030-4633-bfca-026bc93034e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:51.158608 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30109f8d-d030-4633-bfca-026bc93034e0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:51.158895 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30109f8d-d030-4633-bfca-026bc93034e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:51.159168 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30109f8d-d030-4633-bfca-026bc93034e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:51.160016 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30109f8d-d030-4633-bfca-026bc93034e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:51.160259 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30109f8d-d030-4633-bfca-026bc93034e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:51.161729 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30109f8d-d030-4633-bfca-026bc93034e0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:51.185805 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 259/526] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:20:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:51.185805 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30109f8d-d030-4633-bfca-026bc93034e0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:51.201386 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:51.238458 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3693e-9871-4f74-84c2-fa6ad517e593 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:51.274831 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad2325-15d0-4eec-b435-6051e8e85cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:51.276350 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad2325-15d0-4eec-b435-6051e8e85cec None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:51.276833 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad2325-15d0-4eec-b435-6051e8e85cec None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:51.277254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ad2325-15d0-4eec-b435-6051e8e85cec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:51.288398 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30109f8d-d030-4633-bfca-026bc93034e0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:51.288765 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad2325-15d0-4eec-b435-6051e8e85cec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:51.290649 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ad2325-15d0-4eec-b435-6051e8e85cec None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:51.309507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-19ad2325-15d0-4eec-b435-6051e8e85cec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:51.309507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:51.309507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:51.309507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:51.309507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:51.309507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:51.309507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:51.309507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:51.309507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:51.309507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:51.309507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:51.309507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:51.309507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:20:51.309507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:20:51.309507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:20:51.310455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:20:51.310455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:51.310455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:51.310455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:51.310455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:51.310455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:51.310455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:51.310455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:51.310455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:51.310455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:51.310455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:51.310455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:51.310455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:51.310455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:51.310455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:51.310455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:51.310455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:51.311391 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:20:51.311391 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:20:51.311391 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:51.311391 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:51.311391 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 268/527] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:51.312156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad2325-15d0-4eec-b435-6051e8e85cec None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:51.327691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6534c-107e-44ad-9850-7e25dc8ceb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:51.328362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6534c-107e-44ad-9850-7e25dc8ceb31 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:51.328651 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6534c-107e-44ad-9850-7e25dc8ceb31 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:51.328929 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b6534c-107e-44ad-9850-7e25dc8ceb31 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:51.329775 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6534c-107e-44ad-9850-7e25dc8ceb31 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:51.330084 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6534c-107e-44ad-9850-7e25dc8ceb31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:51.331632 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b6534c-107e-44ad-9850-7e25dc8ceb31 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:51.350816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:51.355783 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 260/528] 10.222.0.52 () {56 vars in 1139 bytes} [Wed Jun 29 12:20:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:51.356286 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6534c-107e-44ad-9850-7e25dc8ceb31 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:51.431595 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:51.444342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ad2325-15d0-4eec-b435-6051e8e85cec None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:51.468078 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b6534c-107e-44ad-9850-7e25dc8ceb31 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:51.481250 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a51446-ddfe-4674-b33e-5f13a15d7198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:51.481915 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a51446-ddfe-4674-b33e-5f13a15d7198 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:51.482187 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a51446-ddfe-4674-b33e-5f13a15d7198 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:51.482494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a51446-ddfe-4674-b33e-5f13a15d7198 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:51.483275 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a51446-ddfe-4674-b33e-5f13a15d7198 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:51.483575 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a51446-ddfe-4674-b33e-5f13a15d7198 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:51.485008 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a51446-ddfe-4674-b33e-5f13a15d7198 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:51.502295 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b8d59-112a-4620-9a75-98da2eadb414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:51.502985 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b8d59-112a-4620-9a75-98da2eadb414 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:51.503270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b8d59-112a-4620-9a75-98da2eadb414 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:51.503831 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b8d59-112a-4620-9a75-98da2eadb414 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:51.503966 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 269/529] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:20:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:51.504369 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b8d59-112a-4620-9a75-98da2eadb414 None admin] RBAC: Adding query filter params (role.id=8883c151055d4ba493f514ce1bb62e78, scope.project.id=f043ade9e4f748d4adc010ffb0a47a46, user.id=096234e9f15541c0b187dc8dd527b0fa) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:51.504662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b8d59-112a-4620-9a75-98da2eadb414 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:51.506100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b8d59-112a-4620-9a75-98da2eadb414 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:51.509270 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a51446-ddfe-4674-b33e-5f13a15d7198 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:51.513771 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 261/530] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:20:51 2022] GET /identity/v3/role_assignments?user.id=096234e9f15541c0b187dc8dd527b0fa&scope.project.id=f043ade9e4f748d4adc010ffb0a47a46&role.id=8883c151055d4ba493f514ce1bb62e78 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:51.513974 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b8d59-112a-4620-9a75-98da2eadb414 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:51.628750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:51.636478 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a51446-ddfe-4674-b33e-5f13a15d7198 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:51.658017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b8d59-112a-4620-9a75-98da2eadb414 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:51.678149 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ed2a9-b8b8-4b9e-854a-831297698970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:51.678888 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ed2a9-b8b8-4b9e-854a-831297698970 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:51.679249 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ed2a9-b8b8-4b9e-854a-831297698970 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:51.679643 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ed2a9-b8b8-4b9e-854a-831297698970 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:51.680381 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560ed2a9-b8b8-4b9e-854a-831297698970 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:51.680726 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560ed2a9-b8b8-4b9e-854a-831297698970 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:51.682476 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560ed2a9-b8b8-4b9e-854a-831297698970 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:51.695431 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 270/531] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:20:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:51.696192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ed2a9-b8b8-4b9e-854a-831297698970 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:51.703763 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe5d79-a9b9-4075-b692-e171d5eb72d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:51.704545 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe5d79-a9b9-4075-b692-e171d5eb72d0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:51.704973 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe5d79-a9b9-4075-b692-e171d5eb72d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:51.705365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe5d79-a9b9-4075-b692-e171d5eb72d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:51.708639 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:51.798141 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe5d79-a9b9-4075-b692-e171d5eb72d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:51.799809 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe5d79-a9b9-4075-b692-e171d5eb72d0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:51.816831 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ed2a9-b8b8-4b9e-854a-831297698970 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:51.852058 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2502d639-3403-4849-9a55-8988354c00af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:51.852941 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502d639-3403-4849-9a55-8988354c00af None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:51.853426 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502d639-3403-4849-9a55-8988354c00af None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:51.854018 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502d639-3403-4849-9a55-8988354c00af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:51.855257 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2502d639-3403-4849-9a55-8988354c00af None admin] RBAC: Adding query filter params (role.id=8883c151055d4ba493f514ce1bb62e78, scope.project.id=f043ade9e4f748d4adc010ffb0a47a46, user.id=3ac7391a3c41499d94e5f97478797313) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:51.855781 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2502d639-3403-4849-9a55-8988354c00af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:51.857426 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2502d639-3403-4849-9a55-8988354c00af None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:51.863276 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 271/532] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:20:51 2022] GET /identity/v3/role_assignments?user.id=3ac7391a3c41499d94e5f97478797313&scope.project.id=f043ade9e4f748d4adc010ffb0a47a46&role.id=8883c151055d4ba493f514ce1bb62e78 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:51.864396 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2502d639-3403-4849-9a55-8988354c00af None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:51.882647 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:51.925085 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 262/533] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:20:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:20:51.926666 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe5d79-a9b9-4075-b692-e171d5eb72d0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:51.962700 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:51.982473 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2502d639-3403-4849-9a55-8988354c00af None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:52.020433 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33189fe5-a652-41c1-adb1-e06ceec2c53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:52.021690 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33189fe5-a652-41c1-adb1-e06ceec2c53a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:52.022074 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33189fe5-a652-41c1-adb1-e06ceec2c53a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:52.022494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33189fe5-a652-41c1-adb1-e06ceec2c53a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:52.030983 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33189fe5-a652-41c1-adb1-e06ceec2c53a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:52.032568 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33189fe5-a652-41c1-adb1-e06ceec2c53a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:52.040554 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe5d79-a9b9-4075-b692-e171d5eb72d0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:52.048180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-33189fe5-a652-41c1-adb1-e06ceec2c53a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:20:52.048180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:52.048180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:52.048180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:52.048180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:52.048180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:52.048180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:52.048180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:52.048180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:52.048180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:52.048180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:52.048180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:52.048180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:20:52.048180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:20:52.048180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:20:52.049319 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:20:52.049319 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:52.049319 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:52.049319 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:52.049319 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:52.049319 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:52.049319 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:52.049319 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:52.049319 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:52.049319 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:52.049319 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:52.049319 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:52.049319 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:52.049319 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:52.049319 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:52.049319 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:52.049319 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:52.050278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:20:52.050278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:20:52.050278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:20:52.050278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:20:52.050278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:20:52.050278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:20:52.050278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:20:52.050278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:52.051890 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 272/534] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:20:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:52.059829 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33189fe5-a652-41c1-adb1-e06ceec2c53a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:52.095224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0bdbf2-767d-409d-a43e-ddcac392eba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:52.096481 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bdbf2-767d-409d-a43e-ddcac392eba4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:52.096974 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bdbf2-767d-409d-a43e-ddcac392eba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:52.097449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bdbf2-767d-409d-a43e-ddcac392eba4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:52.117076 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0bdbf2-767d-409d-a43e-ddcac392eba4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:52.118929 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0bdbf2-767d-409d-a43e-ddcac392eba4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:52.141984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9c0bdbf2-767d-409d-a43e-ddcac392eba4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:52.141984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:52.141984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:52.141984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:52.141984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:52.141984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:52.141984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:52.141984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:52.141984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:52.141984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:52.141984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:52.141984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:52.141984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:20:52.141984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:20:52.141984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:20:52.142909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:20:52.142909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:52.142909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:52.142909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:52.142909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:52.142909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:52.142909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:52.142909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:52.142909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:52.142909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:52.142909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:52.142909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:52.142909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:52.142909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:52.142909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:52.142909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:52.142909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:52.143763 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:20:52.143763 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:20:52.143763 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:52.143763 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:52.148979 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:52.150399 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 263/535] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:52.151832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0bdbf2-767d-409d-a43e-ddcac392eba4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:52.178529 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33189fe5-a652-41c1-adb1-e06ceec2c53a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:52.217590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf44e8-2f69-4743-af2d-c1c219b0a7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:52.218665 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf44e8-2f69-4743-af2d-c1c219b0a7cf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:52.219165 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf44e8-2f69-4743-af2d-c1c219b0a7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:52.219654 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaf44e8-2f69-4743-af2d-c1c219b0a7cf None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:52.220631 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaf44e8-2f69-4743-af2d-c1c219b0a7cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:52.221149 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaf44e8-2f69-4743-af2d-c1c219b0a7cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:52.223041 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaf44e8-2f69-4743-af2d-c1c219b0a7cf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:52.236358 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 273/536] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:20:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:52.241537 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf44e8-2f69-4743-af2d-c1c219b0a7cf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:52.250548 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:52.288571 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0bdbf2-767d-409d-a43e-ddcac392eba4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:52.372581 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaf44e8-2f69-4743-af2d-c1c219b0a7cf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:52.407172 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b1d47-f556-428f-8202-e1f796487a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:52.407885 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b1d47-f556-428f-8202-e1f796487a82 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:52.408242 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b1d47-f556-428f-8202-e1f796487a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:52.408530 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b1d47-f556-428f-8202-e1f796487a82 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:52.409377 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b1d47-f556-428f-8202-e1f796487a82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:52.409767 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b1d47-f556-428f-8202-e1f796487a82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:52.411190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98b1d47-f556-428f-8202-e1f796487a82 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:52.412148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045cb35-0a3b-42eb-bd23-904ead0d8e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:52.413475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045cb35-0a3b-42eb-bd23-904ead0d8e3f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:52.413954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045cb35-0a3b-42eb-bd23-904ead0d8e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:52.414461 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045cb35-0a3b-42eb-bd23-904ead0d8e3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:52.415478 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5045cb35-0a3b-42eb-bd23-904ead0d8e3f None admin] RBAC: Adding query filter params (role.id=8883c151055d4ba493f514ce1bb62e78, scope.project.id=f043ade9e4f748d4adc010ffb0a47a46, user.id=356d396ae5cf47c3bc61f24f9e6dfa02) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:52.415983 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5045cb35-0a3b-42eb-bd23-904ead0d8e3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:52.418144 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5045cb35-0a3b-42eb-bd23-904ead0d8e3f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:52.420936 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:52.426569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 274/537] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:20:52 2022] GET /identity/v3/role_assignments?user.id=356d396ae5cf47c3bc61f24f9e6dfa02&scope.project.id=f043ade9e4f748d4adc010ffb0a47a46&role.id=8883c151055d4ba493f514ce1bb62e78 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:52.438843 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 264/538] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:20:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:52.443562 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045cb35-0a3b-42eb-bd23-904ead0d8e3f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:52.559136 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5045cb35-0a3b-42eb-bd23-904ead0d8e3f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:52.601615 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b985f-87e2-486c-9abe-49afcfa234e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:52.602449 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b985f-87e2-486c-9abe-49afcfa234e7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:52.602728 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b985f-87e2-486c-9abe-49afcfa234e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:52.603006 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b985f-87e2-486c-9abe-49afcfa234e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:52.611268 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b985f-87e2-486c-9abe-49afcfa234e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:52.612822 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b985f-87e2-486c-9abe-49afcfa234e7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:52.637875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-467b985f-87e2-486c-9abe-49afcfa234e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:20:52.637875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:52.637875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:52.637875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:52.637875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:52.637875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:52.637875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:52.637875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:52.637875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:52.637875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:52.637875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:52.637875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:52.637875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:20:52.637875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:20:52.637875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:20:52.638839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:20:52.638839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:52.638839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:52.638839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:52.638839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:52.638839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:52.638839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:52.638839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:52.638839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:52.638839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:52.638839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:52.638839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:52.638839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:52.638839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:52.638839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:52.638839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:52.638839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:52.640137 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:20:52.640137 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:20:52.640137 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:20:52.640137 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:20:52.640137 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:20:52.640137 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:20:52.640137 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:20:52.640137 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:52.640137 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 275/539] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:20:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:52.648480 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b1d47-f556-428f-8202-e1f796487a82 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:52.655290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4469ecd-1bb3-4b05-893c-dd34d91beafa None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:52.655580 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4469ecd-1bb3-4b05-893c-dd34d91beafa None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:52.655869 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4469ecd-1bb3-4b05-893c-dd34d91beafa None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:52.660489 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 276/540] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:20:52.662637 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b49285-5541-4e7b-a814-deb4d4cc354a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:52.662919 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b49285-5541-4e7b-a814-deb4d4cc354a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:52.663718 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b49285-5541-4e7b-a814-deb4d4cc354a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:52.684396 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-17b49285-5541-4e7b-a814-deb4d4cc354a None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:52.685514 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-17b49285-5541-4e7b-a814-deb4d4cc354a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:52.741567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:52.756149 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 277/541] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:20:52.781102 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b49285-5541-4e7b-a814-deb4d4cc354a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:52.782535 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b1d47-f556-428f-8202-e1f796487a82 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:52.825410 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe0e2d2-f9f3-4862-b564-eaf751347756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:52.826486 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0e2d2-f9f3-4862-b564-eaf751347756 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:52.826979 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0e2d2-f9f3-4862-b564-eaf751347756 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:52.827453 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe0e2d2-f9f3-4862-b564-eaf751347756 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:52.828443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe0e2d2-f9f3-4862-b564-eaf751347756 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:52.828974 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe0e2d2-f9f3-4862-b564-eaf751347756 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:52.831676 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe0e2d2-f9f3-4862-b564-eaf751347756 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:52.856423 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 265/542] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:20:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:52.863305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe0e2d2-f9f3-4862-b564-eaf751347756 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:52.877632 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:52.941615 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b49285-5541-4e7b-a814-deb4d4cc354a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:52.988336 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735d1d9d-48cc-4fce-95d9-d44eb54253f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:52.989549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d1d9d-48cc-4fce-95d9-d44eb54253f4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:52.990032 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d1d9d-48cc-4fce-95d9-d44eb54253f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:52.990560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735d1d9d-48cc-4fce-95d9-d44eb54253f4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:53.003407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735d1d9d-48cc-4fce-95d9-d44eb54253f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:53.013467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-735d1d9d-48cc-4fce-95d9-d44eb54253f4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:20:53.013467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:53.013467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:53.013467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:53.013467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:53.013467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:53.013467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:53.013467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:53.013467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:53.013467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:53.013467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:53.013467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:53.013467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:53.013467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:53.013467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:53.013467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:53.014359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:53.014359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:53.014359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:53.014359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:53.014359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:53.014359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:53.014359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:53.014359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:53.014359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:53.014359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:53.014359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:53.014359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:53.014359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:53.014359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:53.014359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:53.014359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:53.015210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:53.015210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:53.015210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:53.015210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:53.015210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:53.015210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:53.015210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:53.015210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:53.015210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:53.015210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:53.015210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:20:53.015210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:53.017865 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 278/543] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 12:20:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:53.023510 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735d1d9d-48cc-4fce-95d9-d44eb54253f4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:53.054237 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe0e2d2-f9f3-4862-b564-eaf751347756 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:53.086898 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:53.119093 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e015360-3eb4-4c98-8bd5-2b7b3c355c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:53.119884 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e015360-3eb4-4c98-8bd5-2b7b3c355c04 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:53.120170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e015360-3eb4-4c98-8bd5-2b7b3c355c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:53.120444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e015360-3eb4-4c98-8bd5-2b7b3c355c04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:53.121284 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e015360-3eb4-4c98-8bd5-2b7b3c355c04 None admin] RBAC: Adding query filter params (role.id=8883c151055d4ba493f514ce1bb62e78, scope.project.id=f043ade9e4f748d4adc010ffb0a47a46, user.id=ce922cdbf2d34f10b6e49631f0c6d770) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:53.121598 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e015360-3eb4-4c98-8bd5-2b7b3c355c04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:53.127137 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e015360-3eb4-4c98-8bd5-2b7b3c355c04 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:53.146973 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 266/544] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:20:52 2022] GET /identity/v3/role_assignments?user.id=ce922cdbf2d34f10b6e49631f0c6d770&scope.project.id=f043ade9e4f748d4adc010ffb0a47a46&role.id=8883c151055d4ba493f514ce1bb62e78 => generated 259 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:53.176766 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735d1d9d-48cc-4fce-95d9-d44eb54253f4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:53.243311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0400e-0673-412d-b0aa-87972b1fbd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:53.244023 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0400e-0673-412d-b0aa-87972b1fbd6e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:53.244363 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0400e-0673-412d-b0aa-87972b1fbd6e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:53.244830 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0400e-0673-412d-b0aa-87972b1fbd6e None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:53.245606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0400e-0673-412d-b0aa-87972b1fbd6e None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:53.245922 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0400e-0673-412d-b0aa-87972b1fbd6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:53.247404 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0400e-0673-412d-b0aa-87972b1fbd6e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:53.268877 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 279/545] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 12:20:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:53.274835 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e015360-3eb4-4c98-8bd5-2b7b3c355c04 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:53.410814 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:53.465587 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e015360-3eb4-4c98-8bd5-2b7b3c355c04 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:53.501145 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8ae85-bba1-4370-b251-dc3cc0b80ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:53.501945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8ae85-bba1-4370-b251-dc3cc0b80ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:53.502230 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8ae85-bba1-4370-b251-dc3cc0b80ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:53.502534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8ae85-bba1-4370-b251-dc3cc0b80ad6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:53.514243 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c8ae85-bba1-4370-b251-dc3cc0b80ad6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:53.515737 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c8ae85-bba1-4370-b251-dc3cc0b80ad6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:53.533519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c5c8ae85-bba1-4370-b251-dc3cc0b80ad6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:53.533519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:53.533519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:53.533519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:53.533519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:53.533519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:53.533519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:53.533519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:53.533519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:53.533519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:53.533519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:53.533519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:53.533519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:20:53.533519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:20:53.533519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:20:53.534370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:20:53.534370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:53.534370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:53.534370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:53.534370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:53.534370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:53.534370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:53.534370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:53.534370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:53.534370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:53.534370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:53.534370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:53.534370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:53.534370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:53.534370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:53.534370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:53.534370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:53.535241 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:20:53.535241 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:20:53.535241 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:53.535241 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:53.535241 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 267/546] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:53.543527 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0400e-0673-412d-b0aa-87972b1fbd6e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:53.699599 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0400e-0673-412d-b0aa-87972b1fbd6e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:53.709938 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:53.734949 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b19d7e2-804d-461c-905f-6493ce26c183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:53.735617 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19d7e2-804d-461c-905f-6493ce26c183 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:53.735907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19d7e2-804d-461c-905f-6493ce26c183 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:53.736190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19d7e2-804d-461c-905f-6493ce26c183 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:53.736921 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b19d7e2-804d-461c-905f-6493ce26c183 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:53.737234 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b19d7e2-804d-461c-905f-6493ce26c183 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:53.738670 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b19d7e2-804d-461c-905f-6493ce26c183 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:53.750647 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 280/547] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:20:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:53.756066 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8ae85-bba1-4370-b251-dc3cc0b80ad6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:53.949274 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8ae85-bba1-4370-b251-dc3cc0b80ad6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:53.983401 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:54.001726 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a44f9-ee2d-4f53-a3f2-02629fef793e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:54.002558 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a44f9-ee2d-4f53-a3f2-02629fef793e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:54.002854 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a44f9-ee2d-4f53-a3f2-02629fef793e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:54.003136 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a44f9-ee2d-4f53-a3f2-02629fef793e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:54.014910 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a44f9-ee2d-4f53-a3f2-02629fef793e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:54.016369 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a44f9-ee2d-4f53-a3f2-02629fef793e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:54.036637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0f2a44f9-ee2d-4f53-a3f2-02629fef793e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 12:20:54.036637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:54.036637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:54.036637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:54.036637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:54.036637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:54.036637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:54.036637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:54.036637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:54.036637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:54.036637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:54.036637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:54.036637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:20:54.036637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:20:54.036637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:20:54.036637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:20:54.037657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:54.037657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:54.037657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:20:54.037657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:20:54.037657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:20:54.037657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:20:54.037657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:54.037657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:54.037657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:54.037657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:54.037657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:54.037657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:54.037657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:54.037657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:54.037657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:54.037657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:54.037657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:54.037657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:54.039065 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:54.039065 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:54.039065 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:20:54.039065 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:20:54.039065 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:20:54.039065 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:20:54.039065 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:20:54.039065 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:20:54.039065 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 12:20:54.039065 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:54.039065 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 268/548] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:20:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:54.044299 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b19d7e2-804d-461c-905f-6493ce26c183 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:54.171080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a69c8b-bd68-4ae5-97a7-c48e7c8a74a9 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:54.171447 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a69c8b-bd68-4ae5-97a7-c48e7c8a74a9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:54.171645 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a69c8b-bd68-4ae5-97a7-c48e7c8a74a9 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:54.173020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 269/549] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:20:54.180306 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1075a0-c6e0-4ea2-9f81-efb00a2fb8cc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:54.180601 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1075a0-c6e0-4ea2-9f81-efb00a2fb8cc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:54.180883 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1075a0-c6e0-4ea2-9f81-efb00a2fb8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:54.201105 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b19d7e2-804d-461c-905f-6493ce26c183 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:54.202193 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6b1075a0-c6e0-4ea2-9f81-efb00a2fb8cc None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:54.205387 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6b1075a0-c6e0-4ea2-9f81-efb00a2fb8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:54.256050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af7940-919e-463f-93dc-3ce8d7ad4010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:54.257220 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af7940-919e-463f-93dc-3ce8d7ad4010 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:54.257803 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af7940-919e-463f-93dc-3ce8d7ad4010 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:54.258274 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af7940-919e-463f-93dc-3ce8d7ad4010 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:54.259316 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af7940-919e-463f-93dc-3ce8d7ad4010 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:54.260375 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af7940-919e-463f-93dc-3ce8d7ad4010 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:54.262280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af7940-919e-463f-93dc-3ce8d7ad4010 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:54.278996 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 270/550] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 29 12:20:54.303268 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1075a0-c6e0-4ea2-9f81-efb00a2fb8cc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:54.326582 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 281/551] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Jun 29 12:20:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:54.333228 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af7940-919e-463f-93dc-3ce8d7ad4010 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:54.379836 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:54.430551 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1075a0-c6e0-4ea2-9f81-efb00a2fb8cc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:54.464378 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:54.467432 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb7bde-9e97-4829-b9c0-5c1fa361aa65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:54.469117 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb7bde-9e97-4829-b9c0-5c1fa361aa65 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:54.469582 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb7bde-9e97-4829-b9c0-5c1fa361aa65 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:54.470109 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb7bde-9e97-4829-b9c0-5c1fa361aa65 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:54.475164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af7940-919e-463f-93dc-3ce8d7ad4010 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:54.483506 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cb7bde-9e97-4829-b9c0-5c1fa361aa65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:54.494975 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b0cb7bde-9e97-4829-b9c0-5c1fa361aa65 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:20:54.494975 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:54.494975 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:54.494975 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:54.494975 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:54.494975 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:54.494975 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:54.494975 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:54.494975 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:54.494975 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:54.494975 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:54.494975 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:54.494975 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:54.494975 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:54.494975 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:54.494975 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:54.496055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:54.496055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:54.496055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:54.496055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:54.496055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:54.496055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:54.496055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:54.496055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:54.496055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:54.496055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:54.496055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:54.496055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:54.496055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:54.496055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:54.496055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:54.496055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:54.497182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:54.497182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:54.497182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:54.497182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:54.497182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:54.497182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:54.497182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:54.497182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:54.497182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:54.497182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:54.497182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:20:54.497182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:54.500089 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 271/552] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 12:20:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:20:54.501467 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9085bc8-2dcd-4ddd-85a3-f8cf981da0f4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:54.502405 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9085bc8-2dcd-4ddd-85a3-f8cf981da0f4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:54.502926 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9085bc8-2dcd-4ddd-85a3-f8cf981da0f4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:54.508755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 272/553] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:54.509813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9085bc8-2dcd-4ddd-85a3-f8cf981da0f4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:54.524841 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865dddcc-844a-483c-8814-2da63635ab52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:54.526124 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865dddcc-844a-483c-8814-2da63635ab52 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:54.526416 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865dddcc-844a-483c-8814-2da63635ab52 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:54.526769 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865dddcc-844a-483c-8814-2da63635ab52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:54.626657 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865dddcc-844a-483c-8814-2da63635ab52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:54.629406 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865dddcc-844a-483c-8814-2da63635ab52 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:54.631836 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9085bc8-2dcd-4ddd-85a3-f8cf981da0f4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:54.668356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:54.680041 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680ec4b-dc45-4da3-a5a3-a9000f5a851c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:54.680729 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680ec4b-dc45-4da3-a5a3-a9000f5a851c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:54.681020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680ec4b-dc45-4da3-a5a3-a9000f5a851c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:54.681302 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680ec4b-dc45-4da3-a5a3-a9000f5a851c None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:54.682657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5680ec4b-dc45-4da3-a5a3-a9000f5a851c None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:54.682973 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5680ec4b-dc45-4da3-a5a3-a9000f5a851c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:54.684580 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5680ec4b-dc45-4da3-a5a3-a9000f5a851c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:54.696724 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 273/554] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 12:20:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:54.698500 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3d12c-2a5b-4357-9c16-4f232a13a68b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:54.698930 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3d12c-2a5b-4357-9c16-4f232a13a68b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:54.699371 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced3d12c-2a5b-4357-9c16-4f232a13a68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:54.721968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ced3d12c-2a5b-4357-9c16-4f232a13a68b None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:54.724215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ced3d12c-2a5b-4357-9c16-4f232a13a68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:54.760845 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:54.765573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 282/555] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:20:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 12:20:54.767080 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865dddcc-844a-483c-8814-2da63635ab52 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:54.816851 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 274/556] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:54.817796 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3d12c-2a5b-4357-9c16-4f232a13a68b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:54.876216 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865dddcc-844a-483c-8814-2da63635ab52 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:54.904318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:54.914129 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529a04ad-e84f-4965-9338-f52e71a3aef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:54.915286 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a04ad-e84f-4965-9338-f52e71a3aef5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:54.915786 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a04ad-e84f-4965-9338-f52e71a3aef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:54.916809 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529a04ad-e84f-4965-9338-f52e71a3aef5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:54.927746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529a04ad-e84f-4965-9338-f52e71a3aef5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:54.929470 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529a04ad-e84f-4965-9338-f52e71a3aef5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:54.947148 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced3d12c-2a5b-4357-9c16-4f232a13a68b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:54.952533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-529a04ad-e84f-4965-9338-f52e71a3aef5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:54.952533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:54.952533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:54.952533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:54.952533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:54.952533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:54.952533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:54.952533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:54.952533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:54.952533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:54.952533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:54.952533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:54.952533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:20:54.952533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:20:54.952533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:20:54.953590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:20:54.953590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:54.953590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:54.953590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:54.953590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:54.953590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:54.953590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:54.953590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:54.953590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:54.953590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:54.953590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:54.953590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:54.953590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:54.953590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:54.953590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:54.953590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:54.953590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:54.954740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:20:54.954740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:20:54.954740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:54.954740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:54.955981 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 283/557] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:54.956855 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529a04ad-e84f-4965-9338-f52e71a3aef5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:54.993965 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4796c3e-078f-45e8-b7aa-1930ba7af3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:54.994880 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4796c3e-078f-45e8-b7aa-1930ba7af3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:54.995336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4796c3e-078f-45e8-b7aa-1930ba7af3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:54.995636 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4796c3e-078f-45e8-b7aa-1930ba7af3a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:55.005914 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4796c3e-078f-45e8-b7aa-1930ba7af3a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:55.007413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4796c3e-078f-45e8-b7aa-1930ba7af3a2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:55.027347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d4796c3e-078f-45e8-b7aa-1930ba7af3a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:55.027347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:55.027347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:55.027347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:55.027347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:55.027347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:55.027347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:55.027347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:55.027347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:55.027347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:55.027347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:55.027347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:55.027347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:20:55.027347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:20:55.027347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:20:55.028905 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:20:55.028905 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:55.028905 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:55.028905 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:55.028905 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:55.028905 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:55.028905 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:55.028905 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:55.028905 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:55.028905 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:55.028905 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:55.028905 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:55.028905 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:55.028905 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:55.028905 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:55.028905 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:55.028905 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:55.029979 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:20:55.029979 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:20:55.029979 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:55.029979 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:55.031518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 275/558] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:55.032443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4796c3e-078f-45e8-b7aa-1930ba7af3a2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:55.052119 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:55.085931 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529a04ad-e84f-4965-9338-f52e71a3aef5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:55.128323 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f847c-cf3e-47e0-90ba-8e1833312303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:55.129169 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f847c-cf3e-47e0-90ba-8e1833312303 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:55.129454 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f847c-cf3e-47e0-90ba-8e1833312303 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:55.129728 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f847c-cf3e-47e0-90ba-8e1833312303 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:55.142758 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77f847c-cf3e-47e0-90ba-8e1833312303 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:55.142945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4796c3e-078f-45e8-b7aa-1930ba7af3a2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:55.151381 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c77f847c-cf3e-47e0-90ba-8e1833312303 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:20:55.151381 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:55.151381 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:55.151381 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:55.151381 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:55.151381 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:55.151381 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:55.151381 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:55.151381 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:55.151381 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:55.151381 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:55.151381 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:55.151381 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:55.151381 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:55.151381 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:55.151381 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:55.152606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:55.152606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:55.152606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:55.152606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:55.152606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:55.152606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:55.152606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:55.152606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:55.152606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:55.152606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:55.152606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:55.152606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:55.152606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:55.152606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:55.152606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:55.152606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:55.153519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:55.153519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:55.153519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:55.153519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:55.153519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:55.153519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:55.153519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:55.153519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:55.153519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:55.153519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:55.153519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:20:55.153519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:55.153519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 284/559] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 12:20:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:55.154199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa7e91-091b-45f8-8191-23b7ec11f187 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:55.154199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa7e91-091b-45f8-8191-23b7ec11f187 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:55.154199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa7e91-091b-45f8-8191-23b7ec11f187 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:55.159906 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 285/560] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:55.159906 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afa7e91-091b-45f8-8191-23b7ec11f187 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:55.185124 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad3e18-dd37-48b4-83e9-3af34d5c3931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:55.185804 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad3e18-dd37-48b4-83e9-3af34d5c3931 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:55.186091 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad3e18-dd37-48b4-83e9-3af34d5c3931 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:55.186376 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ad3e18-dd37-48b4-83e9-3af34d5c3931 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:55.187214 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad3e18-dd37-48b4-83e9-3af34d5c3931 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:55.187534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad3e18-dd37-48b4-83e9-3af34d5c3931 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:55.189299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ad3e18-dd37-48b4-83e9-3af34d5c3931 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:55.201672 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 276/561] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:55.202530 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad3e18-dd37-48b4-83e9-3af34d5c3931 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:55.208839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:55.274402 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afa7e91-091b-45f8-8191-23b7ec11f187 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:55.283519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:55.310085 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ad3e18-dd37-48b4-83e9-3af34d5c3931 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:55.312165 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609a1c7-ec83-40a7-be5e-37607e2c6a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:55.313761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609a1c7-ec83-40a7-be5e-37607e2c6a98 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:55.314226 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609a1c7-ec83-40a7-be5e-37607e2c6a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:55.314707 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609a1c7-ec83-40a7-be5e-37607e2c6a98 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:55.315617 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609a1c7-ec83-40a7-be5e-37607e2c6a98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:55.316100 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609a1c7-ec83-40a7-be5e-37607e2c6a98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:55.318059 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609a1c7-ec83-40a7-be5e-37607e2c6a98 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:55.331545 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 286/562] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:55.333017 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b193d-11e9-48ac-bef1-8bae8c292f7f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:55.333301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b193d-11e9-48ac-bef1-8bae8c292f7f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:55.333631 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70b193d-11e9-48ac-bef1-8bae8c292f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:55.352979 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f70b193d-11e9-48ac-bef1-8bae8c292f7f None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:55.354036 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92739700-4065-4f1f-a9c4-385cef6d452f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:55.354464 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f70b193d-11e9-48ac-bef1-8bae8c292f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:55.354755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92739700-4065-4f1f-a9c4-385cef6d452f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:55.355044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92739700-4065-4f1f-a9c4-385cef6d452f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:55.355326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92739700-4065-4f1f-a9c4-385cef6d452f None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:55.356078 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92739700-4065-4f1f-a9c4-385cef6d452f None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:55.356392 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92739700-4065-4f1f-a9c4-385cef6d452f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:55.357787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92739700-4065-4f1f-a9c4-385cef6d452f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:55.376960 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 277/563] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 12:20:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:55.376960 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92739700-4065-4f1f-a9c4-385cef6d452f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:55.437275 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 287/564] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:55.438491 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b193d-11e9-48ac-bef1-8bae8c292f7f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:55.477434 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:55.497504 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92739700-4065-4f1f-a9c4-385cef6d452f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:55.530782 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af65d4b-bcff-4de1-9dae-6de9925a0e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:55.531584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af65d4b-bcff-4de1-9dae-6de9925a0e27 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:55.531873 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af65d4b-bcff-4de1-9dae-6de9925a0e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:55.532155 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af65d4b-bcff-4de1-9dae-6de9925a0e27 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:55.539922 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af65d4b-bcff-4de1-9dae-6de9925a0e27 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:55.541904 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af65d4b-bcff-4de1-9dae-6de9925a0e27 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:55.543838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:55.555687 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70b193d-11e9-48ac-bef1-8bae8c292f7f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:55.558370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3af65d4b-bcff-4de1-9dae-6de9925a0e27 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 29 12:20:55.558370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:55.558370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:55.558370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:55.558370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:55.558370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:55.558370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:55.558370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:55.558370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:55.558370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:55.558370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:55.558370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:55.558370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:20:55.558370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:20:55.558370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:20:55.558370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:20:55.559541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:55.559541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:55.559541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:20:55.559541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:20:55.559541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:20:55.559541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:20:55.559541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:55.559541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:55.559541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:55.559541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:55.559541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:55.559541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:55.559541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:55.559541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:55.559541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:55.559541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:55.559541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:55.559541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:55.560560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:55.560560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:55.560560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:20:55.560560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:20:55.560560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:20:55.560560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:20:55.560560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:20:55.560560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:20:55.560560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 29 12:20:55.560560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:55.560560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 278/565] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:55.561167 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af65d4b-bcff-4de1-9dae-6de9925a0e27 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:55.597772 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaedba4-adf6-45a7-aa91-73cf7d235f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:55.598606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaedba4-adf6-45a7-aa91-73cf7d235f5f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:55.598893 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaedba4-adf6-45a7-aa91-73cf7d235f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:55.599173 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaedba4-adf6-45a7-aa91-73cf7d235f5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:55.606673 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaedba4-adf6-45a7-aa91-73cf7d235f5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:55.608134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaedba4-adf6-45a7-aa91-73cf7d235f5f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:55.622400 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5aaedba4-adf6-45a7-aa91-73cf7d235f5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:20:55.622400 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:55.622400 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:55.622400 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:55.622400 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:55.622400 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:55.622400 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:55.622400 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:55.622400 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:55.622400 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:55.622400 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:55.622400 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:55.622400 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:20:55.622400 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:20:55.622400 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:20:55.623417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:20:55.623417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:55.623417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:55.623417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:55.623417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:55.623417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:55.623417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:55.623417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:55.623417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:55.623417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:55.623417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:55.623417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:55.623417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:55.623417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:55.623417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:55.623417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:55.623417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:55.624675 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:20:55.624675 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:20:55.624675 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:20:55.624675 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:20:55.624675 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:20:55.624675 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:20:55.624675 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:20:55.624675 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:55.624675 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 288/566] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:20:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:55.624675 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaedba4-adf6-45a7-aa91-73cf7d235f5f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:55.665468 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af65d4b-bcff-4de1-9dae-6de9925a0e27 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:55.699139 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1916a-fac1-49d3-a8d8-93dbaf7c4393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:55.699944 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1916a-fac1-49d3-a8d8-93dbaf7c4393 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:55.700226 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1916a-fac1-49d3-a8d8-93dbaf7c4393 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:55.700501 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a1916a-fac1-49d3-a8d8-93dbaf7c4393 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:55.710684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a1916a-fac1-49d3-a8d8-93dbaf7c4393 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:55.711602 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:55.712130 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a1916a-fac1-49d3-a8d8-93dbaf7c4393 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:55.721286 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaedba4-adf6-45a7-aa91-73cf7d235f5f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:55.730067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e0a1916a-fac1-49d3-a8d8-93dbaf7c4393 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:55.730067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:55.730067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:55.730067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:55.730067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:55.730067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:55.730067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:55.730067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:55.730067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:55.730067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:55.730067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:55.730067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:55.730067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:20:55.730067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:20:55.730067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:20:55.731028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:20:55.731028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:55.731028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:55.731028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:55.731028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:55.731028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:55.731028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:55.731028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:55.731028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:55.731028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:55.731028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:55.731028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:55.731028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:55.731028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:55.731028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:55.731028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:55.731028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:55.731934 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:20:55.731934 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:20:55.731934 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:55.731934 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:55.731934 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 279/567] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:55.732529 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1916a-fac1-49d3-a8d8-93dbaf7c4393 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:55.759147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08260b3f-485d-44f7-850d-e30d8b8fe34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:55.760479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08260b3f-485d-44f7-850d-e30d8b8fe34d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:55.760754 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08260b3f-485d-44f7-850d-e30d8b8fe34d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:55.761035 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08260b3f-485d-44f7-850d-e30d8b8fe34d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:55.771589 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08260b3f-485d-44f7-850d-e30d8b8fe34d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:55.779954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-08260b3f-485d-44f7-850d-e30d8b8fe34d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:20:55.779954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:55.779954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:55.779954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:55.779954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:55.779954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:55.779954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:55.779954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:55.779954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:55.779954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:55.779954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:55.779954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:55.779954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:55.779954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:55.779954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:55.779954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:55.781080 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:55.781080 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:55.781080 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:55.781080 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:55.781080 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:55.781080 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:55.781080 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:55.781080 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:55.781080 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:55.781080 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:55.781080 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:55.781080 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:55.781080 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:55.781080 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:55.781080 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:55.781080 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:55.781914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:55.781914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:55.781914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:55.781914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:55.781914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:55.781914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:55.781914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:55.781914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:55.781914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:55.781914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:55.781914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:20:55.781914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:55.781914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 289/568] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 12:20:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:55.782627 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8598271b-d94d-4d78-9308-dceb7beea48e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:55.783094 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8598271b-d94d-4d78-9308-dceb7beea48e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:55.783094 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8598271b-d94d-4d78-9308-dceb7beea48e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:55.783631 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:55.788674 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 290/569] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:55.788674 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8598271b-d94d-4d78-9308-dceb7beea48e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:55.835742 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a1916a-fac1-49d3-a8d8-93dbaf7c4393 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:55.867569 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6332d98-7f58-4a29-bc7e-4e885c74769b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:55.868233 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6332d98-7f58-4a29-bc7e-4e885c74769b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:55.868520 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6332d98-7f58-4a29-bc7e-4e885c74769b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:55.868797 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6332d98-7f58-4a29-bc7e-4e885c74769b None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:55.870280 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6332d98-7f58-4a29-bc7e-4e885c74769b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:55.870619 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6332d98-7f58-4a29-bc7e-4e885c74769b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:55.872051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6332d98-7f58-4a29-bc7e-4e885c74769b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:55.886911 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8598271b-d94d-4d78-9308-dceb7beea48e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:55.888617 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 280/570] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Jun 29 12:20:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:55.889413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6332d98-7f58-4a29-bc7e-4e885c74769b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:55.925498 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44245a67-a4a1-4937-92bb-a1e750f6f25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:55.926157 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44245a67-a4a1-4937-92bb-a1e750f6f25a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:55.926471 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44245a67-a4a1-4937-92bb-a1e750f6f25a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:55.926758 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44245a67-a4a1-4937-92bb-a1e750f6f25a None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:55.927546 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44245a67-a4a1-4937-92bb-a1e750f6f25a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:55.927861 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44245a67-a4a1-4937-92bb-a1e750f6f25a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:55.929310 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44245a67-a4a1-4937-92bb-a1e750f6f25a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:55.941451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 291/571] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:20:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:55.942265 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44245a67-a4a1-4937-92bb-a1e750f6f25a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:55.953653 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:55.989763 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6332d98-7f58-4a29-bc7e-4e885c74769b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:55.999196 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:56.024159 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28668140-0612-4c14-9570-ec4c864bb329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:56.024813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28668140-0612-4c14-9570-ec4c864bb329 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:56.025095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28668140-0612-4c14-9570-ec4c864bb329 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:56.025370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28668140-0612-4c14-9570-ec4c864bb329 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:56.026847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28668140-0612-4c14-9570-ec4c864bb329 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:56.027157 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28668140-0612-4c14-9570-ec4c864bb329 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:56.028507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28668140-0612-4c14-9570-ec4c864bb329 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:56.039332 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 281/572] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:56.040463 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44245a67-a4a1-4937-92bb-a1e750f6f25a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:56.040577 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7adb1-ae13-49ad-b89c-597fbb08ed39 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:56.040775 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7adb1-ae13-49ad-b89c-597fbb08ed39 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:56.041049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7adb1-ae13-49ad-b89c-597fbb08ed39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:56.057256 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6be7adb1-ae13-49ad-b89c-597fbb08ed39 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:56.058445 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6be7adb1-ae13-49ad-b89c-597fbb08ed39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:56.077341 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b51a6-0411-4a92-a216-17deaa5a641d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:56.077994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b51a6-0411-4a92-a216-17deaa5a641d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:56.078276 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b51a6-0411-4a92-a216-17deaa5a641d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:56.078584 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b51a6-0411-4a92-a216-17deaa5a641d None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:56.079320 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b51a6-0411-4a92-a216-17deaa5a641d None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:56.079622 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b51a6-0411-4a92-a216-17deaa5a641d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:56.080975 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b51a6-0411-4a92-a216-17deaa5a641d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:56.089219 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 292/573] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 12:20:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:56.089925 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b51a6-0411-4a92-a216-17deaa5a641d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:56.126944 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 282/574] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:56.127860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7adb1-ae13-49ad-b89c-597fbb08ed39 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:56.161663 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:56.192770 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b51a6-0411-4a92-a216-17deaa5a641d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:56.226018 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0546c3-8d36-44a9-a569-7248f8850239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:56.226836 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0546c3-8d36-44a9-a569-7248f8850239 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:56.227130 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0546c3-8d36-44a9-a569-7248f8850239 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:56.227410 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0546c3-8d36-44a9-a569-7248f8850239 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:56.229501 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:56.233512 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7adb1-ae13-49ad-b89c-597fbb08ed39 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:56.237781 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0546c3-8d36-44a9-a569-7248f8850239 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:56.239279 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0546c3-8d36-44a9-a569-7248f8850239 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:56.256081 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-1b0546c3-8d36-44a9-a569-7248f8850239 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:56.256081 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:56.256081 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:56.256081 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:56.256081 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:56.256081 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:56.256081 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:56.256081 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:56.256081 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:56.256081 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:56.256081 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:56.256081 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:56.256081 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:20:56.256081 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:20:56.256081 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:20:56.257148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:20:56.257148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:56.257148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:56.257148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:56.257148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:56.257148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:56.257148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:56.257148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:56.257148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:56.257148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:56.257148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:56.257148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:56.257148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:56.257148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:56.257148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:56.257148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:56.257148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:56.258057 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:20:56.258057 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:20:56.258057 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:56.258057 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:56.258057 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 293/575] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:56.259235 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0546c3-8d36-44a9-a569-7248f8850239 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:56.270583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71208a32-c438-4747-ba11-5167127fe644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:56.271503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71208a32-c438-4747-ba11-5167127fe644 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:56.271787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71208a32-c438-4747-ba11-5167127fe644 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:56.272060 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71208a32-c438-4747-ba11-5167127fe644 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:56.272898 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71208a32-c438-4747-ba11-5167127fe644 None admin] RBAC: Adding query filter params (role.id=8883c151055d4ba493f514ce1bb62e78, scope.project.id=f043ade9e4f748d4adc010ffb0a47a46, user.id=209b0df45f76404da880a9ae0c637d67) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:56.273211 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71208a32-c438-4747-ba11-5167127fe644 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:56.274670 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71208a32-c438-4747-ba11-5167127fe644 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:56.280629 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 283/576] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:20:56 2022] GET /identity/v3/role_assignments?user.id=209b0df45f76404da880a9ae0c637d67&scope.project.id=f043ade9e4f748d4adc010ffb0a47a46&role.id=8883c151055d4ba493f514ce1bb62e78 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:56.281396 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71208a32-c438-4747-ba11-5167127fe644 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:56.376998 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0546c3-8d36-44a9-a569-7248f8850239 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:56.404310 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:56.412271 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71208a32-c438-4747-ba11-5167127fe644 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:56.421060 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b56ec-f6bd-472e-ba4b-0139fb274eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:56.421974 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b56ec-f6bd-472e-ba4b-0139fb274eae None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:56.422332 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b56ec-f6bd-472e-ba4b-0139fb274eae None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:56.423091 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2b56ec-f6bd-472e-ba4b-0139fb274eae None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:56.432510 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2b56ec-f6bd-472e-ba4b-0139fb274eae None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:56.433697 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2b56ec-f6bd-472e-ba4b-0139fb274eae None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:56.450414 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-4d2b56ec-f6bd-472e-ba4b-0139fb274eae None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 12:20:56.450414 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:56.450414 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:56.450414 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:56.450414 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:56.450414 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:56.450414 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:56.450414 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:56.450414 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:56.450414 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:56.450414 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:56.450414 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:56.450414 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:20:56.450414 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:20:56.450414 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:20:56.450414 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:20:56.451633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:56.451633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:56.451633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:20:56.451633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:20:56.451633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:20:56.451633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:20:56.451633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:56.451633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:56.451633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:56.451633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:56.451633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:56.451633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:56.451633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:56.451633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:56.451633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:56.451633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:56.451633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:56.451633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:56.452793 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:56.452793 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:56.452793 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:20:56.452793 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:20:56.452793 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:20:56.452793 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:20:56.452793 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:20:56.452793 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:20:56.452793 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 12:20:56.452793 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:56.452793 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 294/577] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Jun 29 12:20:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:56.452793 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b56ec-f6bd-472e-ba4b-0139fb274eae None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:56.458166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546c8b7-c7b4-4c86-b487-28fe9429eb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:56.459055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546c8b7-c7b4-4c86-b487-28fe9429eb58 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:56.459345 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546c8b7-c7b4-4c86-b487-28fe9429eb58 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:56.459620 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546c8b7-c7b4-4c86-b487-28fe9429eb58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:56.472093 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a546c8b7-c7b4-4c86-b487-28fe9429eb58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:56.473585 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a546c8b7-c7b4-4c86-b487-28fe9429eb58 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:56.491908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a546c8b7-c7b4-4c86-b487-28fe9429eb58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:56.491908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:56.491908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:56.491908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:56.491908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:56.491908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:56.491908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:56.491908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:56.491908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:56.491908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:56.491908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:56.491908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:56.491908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:20:56.491908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:20:56.491908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:20:56.492802 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:20:56.492802 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:56.492802 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:56.492802 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:56.492802 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:56.492802 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:56.492802 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:56.492802 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:56.492802 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:56.492802 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:56.492802 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:56.492802 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:56.492802 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:56.492802 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:56.492802 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:56.492802 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:56.492802 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:56.493692 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:20:56.493692 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:20:56.493692 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:56.493692 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:56.493692 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 284/578] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:56.493965 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546c8b7-c7b4-4c86-b487-28fe9429eb58 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:56.511301 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:56.563850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2b56ec-f6bd-472e-ba4b-0139fb274eae None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:56.600646 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e5d8d-2308-459a-9a87-e7a5b12abab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:56.601553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e5d8d-2308-459a-9a87-e7a5b12abab6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:56.601839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e5d8d-2308-459a-9a87-e7a5b12abab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:56.602112 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e5d8d-2308-459a-9a87-e7a5b12abab6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:56.606106 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546c8b7-c7b4-4c86-b487-28fe9429eb58 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:56.612985 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e5d8d-2308-459a-9a87-e7a5b12abab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:56.621497 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-dd7e5d8d-2308-459a-9a87-e7a5b12abab6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:20:56.621497 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:56.621497 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:56.621497 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:56.621497 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:56.621497 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:56.621497 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:56.621497 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:56.621497 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:56.621497 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:56.621497 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:56.621497 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:56.621497 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:56.621497 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:56.621497 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:56.621497 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:56.622682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:56.622682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:56.622682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:56.622682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:56.622682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:56.622682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:56.622682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:56.622682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:56.622682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:56.622682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:56.622682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:56.622682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:56.622682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:56.622682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:56.622682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:56.622682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:56.623711 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:56.623711 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:56.623711 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:56.623711 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:56.623711 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:56.623711 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:56.623711 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:56.623711 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:56.623711 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:56.623711 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:56.623711 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:20:56.623711 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:56.627280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 295/579] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 12:20:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:56.628099 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e5d8d-2308-459a-9a87-e7a5b12abab6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:56.645984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6c032-1779-41dd-bdb5-d0a42f10cbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:56.647163 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6c032-1779-41dd-bdb5-d0a42f10cbe0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:56.647592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6c032-1779-41dd-bdb5-d0a42f10cbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:56.648036 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6c032-1779-41dd-bdb5-d0a42f10cbe0 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:56.648904 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6c032-1779-41dd-bdb5-d0a42f10cbe0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:56.649213 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6c032-1779-41dd-bdb5-d0a42f10cbe0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:56.651196 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6c032-1779-41dd-bdb5-d0a42f10cbe0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:56.664675 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 285/580] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:20:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:56.665503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6c032-1779-41dd-bdb5-d0a42f10cbe0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:56.684342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:56.743240 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e5d8d-2308-459a-9a87-e7a5b12abab6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:56.747919 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:56.778045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6c032-1779-41dd-bdb5-d0a42f10cbe0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:56.784985 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b7c01-2391-41e0-8c2b-1b99d024fdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:56.785918 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b7c01-2391-41e0-8c2b-1b99d024fdfc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:56.785918 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b7c01-2391-41e0-8c2b-1b99d024fdfc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:56.786207 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730b7c01-2391-41e0-8c2b-1b99d024fdfc None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:56.787077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b7c01-2391-41e0-8c2b-1b99d024fdfc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:56.787397 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b7c01-2391-41e0-8c2b-1b99d024fdfc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:56.788844 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730b7c01-2391-41e0-8c2b-1b99d024fdfc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:56.808677 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 296/581] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Jun 29 12:20:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:56.809521 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b7c01-2391-41e0-8c2b-1b99d024fdfc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:56.818767 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353df8b1-c36f-49a4-abf2-fbb142e6aad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:56.819443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353df8b1-c36f-49a4-abf2-fbb142e6aad4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:56.819736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353df8b1-c36f-49a4-abf2-fbb142e6aad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:56.820022 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353df8b1-c36f-49a4-abf2-fbb142e6aad4 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:56.820778 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353df8b1-c36f-49a4-abf2-fbb142e6aad4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:56.821093 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353df8b1-c36f-49a4-abf2-fbb142e6aad4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:56.822681 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353df8b1-c36f-49a4-abf2-fbb142e6aad4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:56.839754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 286/582] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:20:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:56.839754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353df8b1-c36f-49a4-abf2-fbb142e6aad4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:56.923247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730b7c01-2391-41e0-8c2b-1b99d024fdfc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:56.925007 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:56.943670 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353df8b1-c36f-49a4-abf2-fbb142e6aad4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:56.965820 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc1dbb0-1911-4234-8bae-c537a1e8499a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:56.966784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1dbb0-1911-4234-8bae-c537a1e8499a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:56.967062 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1dbb0-1911-4234-8bae-c537a1e8499a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:56.967495 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1dbb0-1911-4234-8bae-c537a1e8499a None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:56.968559 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc1dbb0-1911-4234-8bae-c537a1e8499a None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:56.969000 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc1dbb0-1911-4234-8bae-c537a1e8499a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:56.970697 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc1dbb0-1911-4234-8bae-c537a1e8499a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:56.978465 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52f2f4-f1e8-4d1a-9f55-66506e42f608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:56.980403 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52f2f4-f1e8-4d1a-9f55-66506e42f608 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:56.980692 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52f2f4-f1e8-4d1a-9f55-66506e42f608 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:56.980974 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52f2f4-f1e8-4d1a-9f55-66506e42f608 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:56.983223 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:56.985297 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 297/583] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 12:20:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:56.986296 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc1dbb0-1911-4234-8bae-c537a1e8499a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:56.989307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52f2f4-f1e8-4d1a-9f55-66506e42f608 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:56.991193 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52f2f4-f1e8-4d1a-9f55-66506e42f608 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:57.006068 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3a52f2f4-f1e8-4d1a-9f55-66506e42f608 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:20:57.006068 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:57.006068 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:57.006068 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:57.006068 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:57.006068 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:57.006068 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:57.006068 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:57.006068 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:57.006068 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:57.006068 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:57.006068 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:57.006068 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:20:57.006068 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:20:57.006068 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:20:57.007001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:20:57.007001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:57.007001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:57.007001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:57.007001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:57.007001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:57.007001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:57.007001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:57.007001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:57.007001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:57.007001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:57.007001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:57.007001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:57.007001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:57.007001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:57.007001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:57.007001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:57.007946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:20:57.007946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:20:57.007946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:20:57.007946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:20:57.007946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:20:57.007946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:20:57.007946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:20:57.007946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:57.007946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 287/584] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:20:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:57.008443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52f2f4-f1e8-4d1a-9f55-66506e42f608 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:57.105462 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc1dbb0-1911-4234-8bae-c537a1e8499a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:57.118693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52f2f4-f1e8-4d1a-9f55-66506e42f608 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:57.142035 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb112b-4cc0-48ca-b586-108787303741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:57.142847 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb112b-4cc0-48ca-b586-108787303741 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:57.143161 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb112b-4cc0-48ca-b586-108787303741 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:57.143441 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb112b-4cc0-48ca-b586-108787303741 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:57.155326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb112b-4cc0-48ca-b586-108787303741 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:57.155610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2267ea7-628b-4b8d-86de-c92c45a1c843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:57.156402 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2267ea7-628b-4b8d-86de-c92c45a1c843 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:57.156687 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2267ea7-628b-4b8d-86de-c92c45a1c843 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:57.156881 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb112b-4cc0-48ca-b586-108787303741 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:57.156979 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2267ea7-628b-4b8d-86de-c92c45a1c843 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:57.164112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2267ea7-628b-4b8d-86de-c92c45a1c843 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:57.165536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2267ea7-628b-4b8d-86de-c92c45a1c843 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:57.175916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-afeb112b-4cc0-48ca-b586-108787303741 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:57.175916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:57.175916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:57.175916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:57.175916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:57.175916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:57.175916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:57.175916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:57.175916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:57.175916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:57.175916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:57.175916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:57.175916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:20:57.175916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:20:57.175916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:20:57.176925 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:20:57.176925 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:57.176925 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:57.176925 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:57.176925 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:57.176925 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:57.176925 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:57.176925 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:57.176925 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:57.176925 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:57.176925 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:57.176925 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:57.176925 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:57.176925 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:57.176925 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:57.176925 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:57.176925 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:57.177877 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:20:57.177877 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:20:57.177877 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:57.177877 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:57.181300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b2267ea7-628b-4b8d-86de-c92c45a1c843 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 12:20:57.181300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:57.181300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:57.181300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:57.181300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:57.181300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:57.181300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:57.181300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:57.181300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:57.181300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:57.181300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:57.181300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:57.181300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:20:57.181300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:20:57.181300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:20:57.181300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:20:57.182761 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:57.182761 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:57.182761 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:20:57.182761 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:20:57.182761 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:20:57.182761 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:20:57.182761 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:57.182761 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:57.182761 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:57.182761 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:57.182761 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:57.182761 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:57.182761 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:57.182761 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:57.182761 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:57.182761 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:57.182761 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:57.182761 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:57.184895 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:57.184895 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:57.184895 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:20:57.184895 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:20:57.184895 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:20:57.184895 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:20:57.184895 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:20:57.184895 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:20:57.184895 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 12:20:57.184895 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:57.187205 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 298/585] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:57.187205 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb112b-4cc0-48ca-b586-108787303741 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:57.188059 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 288/586] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:20:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:57.188893 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2267ea7-628b-4b8d-86de-c92c45a1c843 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:57.201645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:57.246206 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:57.294176 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb112b-4cc0-48ca-b586-108787303741 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:57.294524 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2267ea7-628b-4b8d-86de-c92c45a1c843 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:57.331550 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc1e7f-5523-4169-bb41-aa04ece65c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:57.331767 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc69c0-b421-4321-ba82-5d23cb7c8999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:57.332205 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc1e7f-5523-4169-bb41-aa04ece65c31 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:57.332374 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc69c0-b421-4321-ba82-5d23cb7c8999 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:57.332478 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc1e7f-5523-4169-bb41-aa04ece65c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:57.332635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc69c0-b421-4321-ba82-5d23cb7c8999 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:57.332741 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc1e7f-5523-4169-bb41-aa04ece65c31 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:57.332905 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc69c0-b421-4321-ba82-5d23cb7c8999 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:57.333515 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc1e7f-5523-4169-bb41-aa04ece65c31 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:57.333824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc1e7f-5523-4169-bb41-aa04ece65c31 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:57.335297 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc1e7f-5523-4169-bb41-aa04ece65c31 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:57.347256 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cc69c0-b421-4321-ba82-5d23cb7c8999 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:57.348747 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cc69c0-b421-4321-ba82-5d23cb7c8999 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:57.351797 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 289/587] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:20:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:57.352860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc1e7f-5523-4169-bb41-aa04ece65c31 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:57.367628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-41cc69c0-b421-4321-ba82-5d23cb7c8999 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:57.367628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:57.367628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:57.367628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:57.367628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:57.367628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:57.367628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:57.367628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:57.367628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:57.367628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:57.367628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:57.367628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:57.367628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:20:57.367628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:20:57.367628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:20:57.368506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:20:57.368506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:57.368506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:57.368506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:57.368506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:57.368506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:57.368506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:57.368506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:57.368506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:57.368506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:57.368506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:57.368506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:57.368506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:57.368506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:57.368506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:57.368506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:57.368506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:57.369639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:20:57.369639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:20:57.369639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:57.369639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:57.373984 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 299/588] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:57.374855 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc69c0-b421-4321-ba82-5d23cb7c8999 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:57.448272 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:57.479594 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc1e7f-5523-4169-bb41-aa04ece65c31 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:57.486041 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc69c0-b421-4321-ba82-5d23cb7c8999 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:57.494110 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:57.516874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b765fbc6-8b88-4dbc-a713-68b7691379ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:57.517850 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765fbc6-8b88-4dbc-a713-68b7691379ed None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:57.518409 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765fbc6-8b88-4dbc-a713-68b7691379ed None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:57.518717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b765fbc6-8b88-4dbc-a713-68b7691379ed None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:57.519442 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b765fbc6-8b88-4dbc-a713-68b7691379ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:57.519752 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b765fbc6-8b88-4dbc-a713-68b7691379ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:57.521171 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b765fbc6-8b88-4dbc-a713-68b7691379ed None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:57.522725 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2eb51-fbf0-464d-83b6-fc9950dff488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:57.523361 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2eb51-fbf0-464d-83b6-fc9950dff488 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:57.523651 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2eb51-fbf0-464d-83b6-fc9950dff488 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:57.523941 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2eb51-fbf0-464d-83b6-fc9950dff488 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:57.524777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa2eb51-fbf0-464d-83b6-fc9950dff488 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:57.525095 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa2eb51-fbf0-464d-83b6-fc9950dff488 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:57.527067 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa2eb51-fbf0-464d-83b6-fc9950dff488 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:57.537572 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 290/589] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:20:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:57.538707 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b765fbc6-8b88-4dbc-a713-68b7691379ed None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:57.549135 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 300/590] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Jun 29 12:20:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:57.549135 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2eb51-fbf0-464d-83b6-fc9950dff488 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:57.648044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b765fbc6-8b88-4dbc-a713-68b7691379ed None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:57.651960 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2eb51-fbf0-464d-83b6-fc9950dff488 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:57.678568 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:57.683706 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d357e4c-f29c-45fc-be5a-d12d2ba0c65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:57.684600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d357e4c-f29c-45fc-be5a-d12d2ba0c65c None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:57.684891 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d357e4c-f29c-45fc-be5a-d12d2ba0c65c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:57.685171 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d357e4c-f29c-45fc-be5a-d12d2ba0c65c None admin] PATH_INFO: `/v3/projects/f043ade9e4f748d4adc010ffb0a47a46/users/096234e9f15541c0b187dc8dd527b0fa/roles/8883c151055d4ba493f514ce1bb62e78` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:57.689895 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97c7c2-9aaf-469f-ad61-8be5ce0af4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:57.690583 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97c7c2-9aaf-469f-ad61-8be5ce0af4d6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:57.690868 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97c7c2-9aaf-469f-ad61-8be5ce0af4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:57.691142 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97c7c2-9aaf-469f-ad61-8be5ce0af4d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:57.691860 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d97c7c2-9aaf-469f-ad61-8be5ce0af4d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:57.692163 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d97c7c2-9aaf-469f-ad61-8be5ce0af4d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:57.693533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d97c7c2-9aaf-469f-ad61-8be5ce0af4d6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:57.700999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d357e4c-f29c-45fc-be5a-d12d2ba0c65c None admin] RBAC: Authorizing `identity:create_grant(project_id=f043ade9e4f748d4adc010ffb0a47a46, user_id=096234e9f15541c0b187dc8dd527b0fa, role_id=8883c151055d4ba493f514ce1bb62e78)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:57.702831 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d357e4c-f29c-45fc-be5a-d12d2ba0c65c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:57.705140 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 301/591] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:20:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:57.705957 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97c7c2-9aaf-469f-ad61-8be5ce0af4d6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:57.719177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 291/592] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 12:20:57 2022] PUT /identity/v3/projects/f043ade9e4f748d4adc010ffb0a47a46/users/096234e9f15541c0b187dc8dd527b0fa/roles/8883c151055d4ba493f514ce1bb62e78 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:20:57.720322 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d357e4c-f29c-45fc-be5a-d12d2ba0c65c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:57.753751 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:57.846348 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97c7c2-9aaf-469f-ad61-8be5ce0af4d6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:57.857116 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d357e4c-f29c-45fc-be5a-d12d2ba0c65c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:57.888525 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c09ad-8905-4b56-8c1d-1460846dce12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:57.889712 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c09ad-8905-4b56-8c1d-1460846dce12 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:57.890218 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c09ad-8905-4b56-8c1d-1460846dce12 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:57.890747 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c09ad-8905-4b56-8c1d-1460846dce12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:57.898341 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648c09ad-8905-4b56-8c1d-1460846dce12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:57.899848 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648c09ad-8905-4b56-8c1d-1460846dce12 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:57.906057 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b86bb1-41e4-47a1-932a-23091b2ac874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:57.906922 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b86bb1-41e4-47a1-932a-23091b2ac874 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:57.907212 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b86bb1-41e4-47a1-932a-23091b2ac874 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:57.907491 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b86bb1-41e4-47a1-932a-23091b2ac874 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:57.918778 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-648c09ad-8905-4b56-8c1d-1460846dce12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:20:57.918778 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:57.918778 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:57.918778 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:57.918778 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:57.918778 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:57.918778 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:57.918778 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:57.918778 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:57.918778 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:57.918778 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:57.918778 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:57.918778 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:20:57.918778 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:20:57.918778 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:20:57.919680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:20:57.919680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:57.919680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:57.919680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:57.919680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:57.919680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:57.919680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:57.919680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:57.919680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:57.919680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:57.919680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:57.919680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:57.919680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:57.919680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:57.919680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:57.919680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:57.919680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:57.920566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:20:57.920566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:20:57.920566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:20:57.920566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:20:57.920566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:20:57.920566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:20:57.920566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:20:57.920566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:57.921872 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b86bb1-41e4-47a1-932a-23091b2ac874 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:57.922994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 302/593] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:20:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:57.923444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b86bb1-41e4-47a1-932a-23091b2ac874 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:57.924333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c09ad-8905-4b56-8c1d-1460846dce12 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:57.944778 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-08b86bb1-41e4-47a1-932a-23091b2ac874 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:57.944778 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:57.944778 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:57.944778 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:57.944778 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:57.944778 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:57.944778 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:57.944778 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:57.944778 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:57.944778 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:57.944778 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:57.944778 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:57.944778 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:20:57.944778 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:20:57.944778 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:20:57.945745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:20:57.945745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:57.945745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:57.945745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:57.945745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:57.945745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:57.945745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:57.945745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:57.945745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:57.945745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:57.945745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:57.945745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:57.945745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:57.945745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:57.945745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:57.945745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:57.945745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:57.946650 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:20:57.946650 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:20:57.946650 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:57.946650 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:57.946650 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 292/594] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:57.947333 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b86bb1-41e4-47a1-932a-23091b2ac874 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:57.983286 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:58.032573 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:58.052248 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c09ad-8905-4b56-8c1d-1460846dce12 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:58.062095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b86bb1-41e4-47a1-932a-23091b2ac874 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:58.089037 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ebcd5-f336-447e-b7bf-7e910738d3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:58.089985 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ebcd5-f336-447e-b7bf-7e910738d3cd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:58.090369 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ebcd5-f336-447e-b7bf-7e910738d3cd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:58.090764 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ebcd5-f336-447e-b7bf-7e910738d3cd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:58.098332 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55ebcd5-f336-447e-b7bf-7e910738d3cd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:58.099899 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55ebcd5-f336-447e-b7bf-7e910738d3cd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:58.100168 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee021911-0e3f-404b-a3a3-31d9be084ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:58.100837 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee021911-0e3f-404b-a3a3-31d9be084ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:58.101126 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee021911-0e3f-404b-a3a3-31d9be084ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:58.101408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee021911-0e3f-404b-a3a3-31d9be084ae0 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:58.102205 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee021911-0e3f-404b-a3a3-31d9be084ae0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:58.102550 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee021911-0e3f-404b-a3a3-31d9be084ae0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:58.104270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee021911-0e3f-404b-a3a3-31d9be084ae0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:58.115621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f55ebcd5-f336-447e-b7bf-7e910738d3cd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 12:20:58.115621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:58.115621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:58.115621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:58.115621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:58.115621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:58.115621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:58.115621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:58.115621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:58.115621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:58.115621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:58.115621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:58.115621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:20:58.115621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:20:58.115621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:20:58.115621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:20:58.117407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:58.117407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:58.117407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:20:58.117407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:20:58.117407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:20:58.117407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:20:58.117407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:58.117407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:58.117407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:58.117407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:58.117407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:58.117407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:58.117407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:58.117407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:58.117407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:58.117407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:58.117407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:58.117407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:58.119064 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:58.119064 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:58.119064 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:20:58.119064 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:20:58.119064 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:20:58.119064 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:20:58.119064 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:20:58.119064 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:20:58.119064 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 12:20:58.119064 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:58.120536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 293/595] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:20:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:58.120536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee021911-0e3f-404b-a3a3-31d9be084ae0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:58.120692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 303/596] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 12:20:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:58.128064 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ebcd5-f336-447e-b7bf-7e910738d3cd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:58.227295 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee021911-0e3f-404b-a3a3-31d9be084ae0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:58.232688 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ebcd5-f336-447e-b7bf-7e910738d3cd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:58.240600 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:58.267075 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7fcc8-b927-4c01-8878-ff2fd7e09d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:58.268152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7fcc8-b927-4c01-8878-ff2fd7e09d79 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:58.268449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7fcc8-b927-4c01-8878-ff2fd7e09d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:58.268736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7fcc8-b927-4c01-8878-ff2fd7e09d79 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:58.269777 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef7fcc8-b927-4c01-8878-ff2fd7e09d79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:58.270092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef7fcc8-b927-4c01-8878-ff2fd7e09d79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:58.271764 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef7fcc8-b927-4c01-8878-ff2fd7e09d79 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:58.274068 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b8e3a-7c52-44d6-8fea-e8bb94fb6563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:58.274987 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b8e3a-7c52-44d6-8fea-e8bb94fb6563 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:58.275518 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b8e3a-7c52-44d6-8fea-e8bb94fb6563 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:58.275990 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b8e3a-7c52-44d6-8fea-e8bb94fb6563 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:58.277194 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75b8e3a-7c52-44d6-8fea-e8bb94fb6563 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:58.277717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75b8e3a-7c52-44d6-8fea-e8bb94fb6563 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:58.279390 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75b8e3a-7c52-44d6-8fea-e8bb94fb6563 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:58.283455 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:58.294260 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 294/597] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:58.294260 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7fcc8-b927-4c01-8878-ff2fd7e09d79 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:58.305835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 304/598] 10.222.0.52 () {56 vars in 1139 bytes} [Wed Jun 29 12:20:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:58.307255 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b8e3a-7c52-44d6-8fea-e8bb94fb6563 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:58.419466 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b8e3a-7c52-44d6-8fea-e8bb94fb6563 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:58.420838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7fcc8-b927-4c01-8878-ff2fd7e09d79 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:58.457764 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee2d73-eab1-4523-add8-105fe38c4c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:58.458605 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee2d73-eab1-4523-add8-105fe38c4c67 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:58.458889 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee2d73-eab1-4523-add8-105fe38c4c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:58.459165 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee2d73-eab1-4523-add8-105fe38c4c67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:58.460116 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce1387-7e1c-4a37-976a-9ac26748137c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:58.461218 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce1387-7e1c-4a37-976a-9ac26748137c None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:58.461699 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce1387-7e1c-4a37-976a-9ac26748137c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:58.461971 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce1387-7e1c-4a37-976a-9ac26748137c None admin] PATH_INFO: `/v3/projects/f043ade9e4f748d4adc010ffb0a47a46/users/3ac7391a3c41499d94e5f97478797313/roles/8883c151055d4ba493f514ce1bb62e78` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:58.466689 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee2d73-eab1-4523-add8-105fe38c4c67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:58.468134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee2d73-eab1-4523-add8-105fe38c4c67 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:58.478109 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce1387-7e1c-4a37-976a-9ac26748137c None admin] RBAC: Authorizing `identity:create_grant(project_id=f043ade9e4f748d4adc010ffb0a47a46, user_id=3ac7391a3c41499d94e5f97478797313, role_id=8883c151055d4ba493f514ce1bb62e78)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:58.479636 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ce1387-7e1c-4a37-976a-9ac26748137c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:58.482292 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-2aee2d73-eab1-4523-add8-105fe38c4c67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:20:58.482292 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:58.482292 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:58.482292 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:58.482292 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:58.482292 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:58.482292 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:58.482292 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:58.482292 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:58.482292 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:58.482292 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:58.482292 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:58.482292 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:20:58.482292 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:20:58.482292 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:20:58.483475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:20:58.483475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:58.483475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:58.483475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:58.483475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:58.483475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:58.483475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:58.483475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:58.483475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:58.483475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:58.483475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:58.483475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:58.483475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:58.483475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:58.483475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:58.483475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:58.483475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:58.485058 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:20:58.485058 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:20:58.485058 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:20:58.485058 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:20:58.485058 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:20:58.485058 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:20:58.485058 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:20:58.485058 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:58.485058 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 305/599] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:20:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:58.485058 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee2d73-eab1-4523-add8-105fe38c4c67 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:58.497300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 295/600] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 12:20:58 2022] PUT /identity/v3/projects/f043ade9e4f748d4adc010ffb0a47a46/users/3ac7391a3c41499d94e5f97478797313/roles/8883c151055d4ba493f514ce1bb62e78 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:20:58.498560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431df31b-020e-4eb9-a122-d011b9eadfef None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:58.498851 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431df31b-020e-4eb9-a122-d011b9eadfef None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:58.499123 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431df31b-020e-4eb9-a122-d011b9eadfef None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:58.501039 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 296/601] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:20:58.505748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431df31b-020e-4eb9-a122-d011b9eadfef None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:58.523651 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:58.568845 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:58.628110 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee2d73-eab1-4523-add8-105fe38c4c67 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:58.659245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431df31b-020e-4eb9-a122-d011b9eadfef None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:58.686787 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2082f-0754-4e2e-99c1-6a4f37224f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:58.688051 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2082f-0754-4e2e-99c1-6a4f37224f32 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:58.688345 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2082f-0754-4e2e-99c1-6a4f37224f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:58.688627 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2082f-0754-4e2e-99c1-6a4f37224f32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:58.701224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2082f-0754-4e2e-99c1-6a4f37224f32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:58.702735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2082f-0754-4e2e-99c1-6a4f37224f32 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:58.703262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd134cf0-85e0-4937-b9ff-07fac4f4de8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:58.704114 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd134cf0-85e0-4937-b9ff-07fac4f4de8d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:58.704410 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd134cf0-85e0-4937-b9ff-07fac4f4de8d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:58.704696 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd134cf0-85e0-4937-b9ff-07fac4f4de8d None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:58.705557 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd134cf0-85e0-4937-b9ff-07fac4f4de8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:58.705880 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd134cf0-85e0-4937-b9ff-07fac4f4de8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:58.707519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd134cf0-85e0-4937-b9ff-07fac4f4de8d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:58.720285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 297/602] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:20:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:58.720448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-95f2082f-0754-4e2e-99c1-6a4f37224f32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:58.720448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:58.720448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:58.720448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:58.720448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:58.720448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:58.720448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:58.720448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:58.720448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:58.720448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:58.720448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:58.720448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:58.720448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:20:58.720448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:20:58.720448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:20:58.721610 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:20:58.721610 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:58.721610 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:58.721610 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:58.721610 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:58.721610 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:58.721610 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:58.721610 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:58.721610 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:58.721610 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:58.721610 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:58.721610 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:58.721610 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:58.721610 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:58.721610 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:58.721610 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:58.721610 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:58.723054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:20:58.723054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:20:58.723054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:58.723054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:58.723054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 306/603] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:58.726583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a627c-01f2-4e08-9ca8-009eae76adcf None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:58.726891 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a627c-01f2-4e08-9ca8-009eae76adcf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:58.727150 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2082f-0754-4e2e-99c1-6a4f37224f32 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:58.727262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a627c-01f2-4e08-9ca8-009eae76adcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:58.749440 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-173a627c-01f2-4e08-9ca8-009eae76adcf None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:20:58.750984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-173a627c-01f2-4e08-9ca8-009eae76adcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:20:58.812109 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:58.828835 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 298/604] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:20:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:20:58.830085 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a627c-01f2-4e08-9ca8-009eae76adcf None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:58.842465 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2082f-0754-4e2e-99c1-6a4f37224f32 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:58.867129 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:58.904739 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc378e1-b436-4b6b-9c8b-756ff768fee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:58.905663 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc378e1-b436-4b6b-9c8b-756ff768fee0 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:58.905950 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc378e1-b436-4b6b-9c8b-756ff768fee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:58.906233 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc378e1-b436-4b6b-9c8b-756ff768fee0 None admin] PATH_INFO: `/v3/projects/f043ade9e4f748d4adc010ffb0a47a46/users/356d396ae5cf47c3bc61f24f9e6dfa02/roles/8883c151055d4ba493f514ce1bb62e78` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:58.923517 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc378e1-b436-4b6b-9c8b-756ff768fee0 None admin] RBAC: Authorizing `identity:create_grant(project_id=f043ade9e4f748d4adc010ffb0a47a46, user_id=356d396ae5cf47c3bc61f24f9e6dfa02, role_id=8883c151055d4ba493f514ce1bb62e78)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:58.925075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc378e1-b436-4b6b-9c8b-756ff768fee0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:58.943514 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a627c-01f2-4e08-9ca8-009eae76adcf None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:58.944341 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 307/605] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 12:20:58 2022] PUT /identity/v3/projects/f043ade9e4f748d4adc010ffb0a47a46/users/356d396ae5cf47c3bc61f24f9e6dfa02/roles/8883c151055d4ba493f514ce1bb62e78 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:20:58.945271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc378e1-b436-4b6b-9c8b-756ff768fee0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:58.988046 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee701e3-1401-4b7d-aa1c-500a61400463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:58.989162 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee701e3-1401-4b7d-aa1c-500a61400463 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:58.989691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee701e3-1401-4b7d-aa1c-500a61400463 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:58.990171 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee701e3-1401-4b7d-aa1c-500a61400463 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:58.991218 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee701e3-1401-4b7d-aa1c-500a61400463 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:58.991757 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee701e3-1401-4b7d-aa1c-500a61400463 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:58.993506 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee701e3-1401-4b7d-aa1c-500a61400463 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:59.007714 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 299/606] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:59.020772 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee701e3-1401-4b7d-aa1c-500a61400463 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:59.087457 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc378e1-b436-4b6b-9c8b-756ff768fee0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:59.118775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:59.130695 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659e271-e168-4bba-a49e-54890f28352d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:59.131501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659e271-e168-4bba-a49e-54890f28352d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:59.131784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659e271-e168-4bba-a49e-54890f28352d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:59.132061 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5659e271-e168-4bba-a49e-54890f28352d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:59.145548 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5659e271-e168-4bba-a49e-54890f28352d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:20:59.155199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5659e271-e168-4bba-a49e-54890f28352d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:20:59.155199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:59.155199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:59.155199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:59.155199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:59.155199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:59.155199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:59.155199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:59.155199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:59.155199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:59.155199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:59.155199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:59.155199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:20:59.155199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:20:59.155199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:20:59.155199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:20:59.156192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:20:59.156192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:20:59.156192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:20:59.156192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:20:59.156192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:59.156192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:59.156192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:59.156192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:59.156192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:59.156192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:59.156192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:59.156192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:59.156192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:59.156192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:59.156192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:59.156192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:59.157446 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:59.157446 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:59.157446 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:59.157446 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:59.157446 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:20:59.157446 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:20:59.157446 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:20:59.157446 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:20:59.157446 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:20:59.157446 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:20:59.157446 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:20:59.157446 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:59.157446 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 308/607] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 12:20:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:59.163272 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659e271-e168-4bba-a49e-54890f28352d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:59.245263 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:59.253023 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee701e3-1401-4b7d-aa1c-500a61400463 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:59.280577 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5659e271-e168-4bba-a49e-54890f28352d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:59.301323 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24294ed9-7b50-4dee-b0f3-9c16614a146d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:59.302126 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24294ed9-7b50-4dee-b0f3-9c16614a146d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:59.302409 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24294ed9-7b50-4dee-b0f3-9c16614a146d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:59.302720 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24294ed9-7b50-4dee-b0f3-9c16614a146d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:59.314843 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24294ed9-7b50-4dee-b0f3-9c16614a146d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:59.316907 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24294ed9-7b50-4dee-b0f3-9c16614a146d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:59.320645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0e5c30-8d08-478d-914a-485638e8f3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:59.321556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e5c30-8d08-478d-914a-485638e8f3d8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:59.322051 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e5c30-8d08-478d-914a-485638e8f3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:59.322752 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e5c30-8d08-478d-914a-485638e8f3d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:59.323955 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0e5c30-8d08-478d-914a-485638e8f3d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:59.324475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0e5c30-8d08-478d-914a-485638e8f3d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:59.326902 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0e5c30-8d08-478d-914a-485638e8f3d8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:59.333428 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-24294ed9-7b50-4dee-b0f3-9c16614a146d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:20:59.333428 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:59.333428 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:59.333428 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:59.333428 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:59.333428 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:59.333428 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:59.333428 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:59.333428 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:59.333428 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:59.333428 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:59.333428 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:59.333428 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:20:59.333428 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:20:59.333428 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:20:59.334407 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:20:59.334407 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:59.334407 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:59.334407 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:59.334407 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:59.334407 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:59.334407 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:59.334407 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:59.334407 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:59.334407 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:59.334407 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:59.334407 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:59.334407 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:59.334407 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:59.334407 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:59.334407 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:59.334407 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:59.335753 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:20:59.335753 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:20:59.335753 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:20:59.335753 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:20:59.335753 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:20:59.335753 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:20:59.335753 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:20:59.335753 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:20:59.335753 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 300/608] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:20:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:59.340917 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 309/609] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 12:20:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:59.341199 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24294ed9-7b50-4dee-b0f3-9c16614a146d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:59.362908 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0e5c30-8d08-478d-914a-485638e8f3d8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:59.466333 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24294ed9-7b50-4dee-b0f3-9c16614a146d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:59.482798 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:59.508584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb48d2a-9d61-42a0-86cc-e84f9b0c8964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:59.509408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb48d2a-9d61-42a0-86cc-e84f9b0c8964 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:59.509705 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb48d2a-9d61-42a0-86cc-e84f9b0c8964 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:59.510667 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb48d2a-9d61-42a0-86cc-e84f9b0c8964 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:59.511797 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb48d2a-9d61-42a0-86cc-e84f9b0c8964 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:59.512118 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb48d2a-9d61-42a0-86cc-e84f9b0c8964 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:59.513699 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb48d2a-9d61-42a0-86cc-e84f9b0c8964 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:59.528274 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 301/610] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:20:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:20:59.533970 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb48d2a-9d61-42a0-86cc-e84f9b0c8964 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:59.578878 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:59.598763 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0e5c30-8d08-478d-914a-485638e8f3d8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:59.646079 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb48d2a-9d61-42a0-86cc-e84f9b0c8964 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:59.671697 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8681850-d52b-4015-b7e3-02c14f9fb75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:59.672508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8681850-d52b-4015-b7e3-02c14f9fb75c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:59.672799 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8681850-d52b-4015-b7e3-02c14f9fb75c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:59.673081 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8681850-d52b-4015-b7e3-02c14f9fb75c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:59.689863 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8681850-d52b-4015-b7e3-02c14f9fb75c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:59.691520 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8681850-d52b-4015-b7e3-02c14f9fb75c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:59.692913 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16da1b7-4fee-4b2d-ae8e-6af79355c58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:59.693921 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16da1b7-4fee-4b2d-ae8e-6af79355c58c None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:59.694212 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16da1b7-4fee-4b2d-ae8e-6af79355c58c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:59.694531 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16da1b7-4fee-4b2d-ae8e-6af79355c58c None admin] PATH_INFO: `/v3/projects/f043ade9e4f748d4adc010ffb0a47a46/users/ce922cdbf2d34f10b6e49631f0c6d770/roles/8883c151055d4ba493f514ce1bb62e78` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:59.709314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c8681850-d52b-4015-b7e3-02c14f9fb75c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:59.709314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:20:59.709314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:20:59.709314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:20:59.709314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:20:59.709314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:20:59.709314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:20:59.709314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:20:59.709314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:20:59.709314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:20:59.709314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:20:59.709314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:20:59.709314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:20:59.709314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:20:59.709314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:20:59.710281 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:20:59.710281 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:20:59.710281 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:20:59.710281 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:20:59.710281 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:20:59.710281 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:20:59.710281 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:20:59.710281 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:20:59.710281 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:20:59.710281 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:20:59.710281 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:20:59.710281 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:20:59.710281 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:20:59.710281 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:20:59.710281 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:20:59.710281 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:20:59.710281 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:20:59.711198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:20:59.711198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:20:59.711198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:20:59.711198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:20:59.711198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 310/611] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:20:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:20:59.723250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16da1b7-4fee-4b2d-ae8e-6af79355c58c None admin] RBAC: Authorizing `identity:create_grant(project_id=f043ade9e4f748d4adc010ffb0a47a46, user_id=ce922cdbf2d34f10b6e49631f0c6d770, role_id=8883c151055d4ba493f514ce1bb62e78)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:59.724813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16da1b7-4fee-4b2d-ae8e-6af79355c58c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:59.727441 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8681850-d52b-4015-b7e3-02c14f9fb75c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:20:59.752204 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 302/612] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 12:20:59 2022] PUT /identity/v3/projects/f043ade9e4f748d4adc010ffb0a47a46/users/ce922cdbf2d34f10b6e49631f0c6d770/roles/8883c151055d4ba493f514ce1bb62e78 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:20:59.847531 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:20:59.854862 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8681850-d52b-4015-b7e3-02c14f9fb75c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:20:59.900625 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da561f-2242-4522-8b6a-eab5e91cadb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:20:59.901568 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da561f-2242-4522-8b6a-eab5e91cadb4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:20:59.901835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da561f-2242-4522-8b6a-eab5e91cadb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:20:59.902119 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da561f-2242-4522-8b6a-eab5e91cadb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:20:59.902897 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da561f-2242-4522-8b6a-eab5e91cadb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:20:59.903211 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da561f-2242-4522-8b6a-eab5e91cadb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:20:59.904630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da561f-2242-4522-8b6a-eab5e91cadb4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:20:59.918877 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 311/613] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:20:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:20:59.924126 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16da1b7-4fee-4b2d-ae8e-6af79355c58c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:00.020665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:00.020875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f59d77-ab03-4bd1-81a5-d6741f6130bb None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:00.021101 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f59d77-ab03-4bd1-81a5-d6741f6130bb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:00.021589 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f59d77-ab03-4bd1-81a5-d6741f6130bb None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:00.023245 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 312/614] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:00.032245 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a3631-8d8c-4460-82d1-3b3f6b7019b0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:00.032533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a3631-8d8c-4460-82d1-3b3f6b7019b0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:00.032810 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a3631-8d8c-4460-82d1-3b3f6b7019b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:00.065720 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16da1b7-4fee-4b2d-ae8e-6af79355c58c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:00.067407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a00a3631-8d8c-4460-82d1-3b3f6b7019b0 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:00.068598 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a00a3631-8d8c-4460-82d1-3b3f6b7019b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:00.139397 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f5fd8-14ed-4b10-96a4-a59e30525672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:00.140149 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f5fd8-14ed-4b10-96a4-a59e30525672 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:00.140446 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f5fd8-14ed-4b10-96a4-a59e30525672 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:00.140735 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f5fd8-14ed-4b10-96a4-a59e30525672 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:00.140847 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 313/615] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:21:00.153341 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f5fd8-14ed-4b10-96a4-a59e30525672 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:00.154852 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f5fd8-14ed-4b10-96a4-a59e30525672 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:00.183194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-424f5fd8-14ed-4b10-96a4-a59e30525672 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 12:21:00.183194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:00.183194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:00.183194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:00.183194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:00.183194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:00.183194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:00.183194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:00.183194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:00.183194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:00.183194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:00.183194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:00.183194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:21:00.183194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:21:00.183194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:21:00.183194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:21:00.184244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:00.184244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:00.184244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:21:00.184244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:00.184244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:21:00.184244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:00.184244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:00.184244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:00.184244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:00.184244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:00.184244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:00.184244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:00.184244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:00.184244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:00.184244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:00.184244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:00.184244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:00.184244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:00.185208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:00.185208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:00.185208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:21:00.185208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:21:00.185208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:21:00.185208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:21:00.185208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:21:00.185208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:21:00.185208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 12:21:00.185208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:00.185208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 303/616] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:20:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:00.190305 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a3631-8d8c-4460-82d1-3b3f6b7019b0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:00.196600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f5fd8-14ed-4b10-96a4-a59e30525672 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:00.292078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:00.298975 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a3631-8d8c-4460-82d1-3b3f6b7019b0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:00.367907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14f6c9-0c1e-4f64-b4fc-6d941f509988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:00.368600 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14f6c9-0c1e-4f64-b4fc-6d941f509988 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:00.368886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14f6c9-0c1e-4f64-b4fc-6d941f509988 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:00.369199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14f6c9-0c1e-4f64-b4fc-6d941f509988 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:00.370045 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14f6c9-0c1e-4f64-b4fc-6d941f509988 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:00.370355 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14f6c9-0c1e-4f64-b4fc-6d941f509988 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:00.371822 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14f6c9-0c1e-4f64-b4fc-6d941f509988 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:00.402883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 314/617] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Jun 29 12:21:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:00.410939 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14f6c9-0c1e-4f64-b4fc-6d941f509988 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:00.422806 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f5fd8-14ed-4b10-96a4-a59e30525672 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:00.464704 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:00.484977 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d5a06-ffeb-4dc9-9a17-efcf1cd666fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:00.485786 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d5a06-ffeb-4dc9-9a17-efcf1cd666fa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:00.486073 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d5a06-ffeb-4dc9-9a17-efcf1cd666fa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:00.486353 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d5a06-ffeb-4dc9-9a17-efcf1cd666fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:00.506551 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d5a06-ffeb-4dc9-9a17-efcf1cd666fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:21:00.515254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ea3d5a06-ffeb-4dc9-9a17-efcf1cd666fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:21:00.515254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:00.515254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:00.515254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:00.515254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:00.515254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:00.515254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:00.515254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:00.515254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:00.515254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:00.515254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:00.515254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:00.515254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:21:00.515254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:21:00.515254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:21:00.515254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:21:00.516178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:21:00.516178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:21:00.516178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:21:00.516178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:21:00.516178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:00.516178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:00.516178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:00.516178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:00.516178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:00.516178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:00.516178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:00.516178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:00.516178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:00.516178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:00.516178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:00.516178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:00.517019 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:00.517019 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:00.517019 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:00.517019 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:00.517019 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:21:00.517019 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:21:00.517019 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:21:00.517019 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:21:00.517019 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:21:00.517019 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:21:00.517019 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:21:00.517019 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:00.517019 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 304/618] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 12:21:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:00.527399 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d5a06-ffeb-4dc9-9a17-efcf1cd666fa None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:00.593923 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14f6c9-0c1e-4f64-b4fc-6d941f509988 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:00.660631 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d5eb5d-804f-450b-ac55-0ec477e3ebd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:00.661346 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d5eb5d-804f-450b-ac55-0ec477e3ebd6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:00.661630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d5eb5d-804f-450b-ac55-0ec477e3ebd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:00.661906 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d5eb5d-804f-450b-ac55-0ec477e3ebd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:00.685551 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d5eb5d-804f-450b-ac55-0ec477e3ebd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:00.687326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d5eb5d-804f-450b-ac55-0ec477e3ebd6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:00.706562 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:00.719474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d1d5eb5d-804f-450b-ac55-0ec477e3ebd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:00.719474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:00.719474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:00.719474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:00.719474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:00.719474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:00.719474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:00.719474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:00.719474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:00.719474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:00.719474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:00.719474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:00.719474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:00.719474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:00.719474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:00.720392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:00.720392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:00.720392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:00.720392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:00.720392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:00.720392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:00.720392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:00.720392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:00.720392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:00.720392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:00.720392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:00.720392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:00.720392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:00.720392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:00.720392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:00.720392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:00.720392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:00.721437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:00.721437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:00.721437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:00.721437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:00.721755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 315/619] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:00.732966 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d5a06-ffeb-4dc9-9a17-efcf1cd666fa None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:00.734583 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d5eb5d-804f-450b-ac55-0ec477e3ebd6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:00.793759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d6df58-40c2-4544-a98f-9dcfbca73a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:00.794446 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6df58-40c2-4544-a98f-9dcfbca73a60 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:00.794730 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6df58-40c2-4544-a98f-9dcfbca73a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:00.795006 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6df58-40c2-4544-a98f-9dcfbca73a60 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:00.795760 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d6df58-40c2-4544-a98f-9dcfbca73a60 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:00.796287 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d6df58-40c2-4544-a98f-9dcfbca73a60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:00.799663 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d6df58-40c2-4544-a98f-9dcfbca73a60 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:00.817279 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 305/620] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 12:21:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:00.821767 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d6df58-40c2-4544-a98f-9dcfbca73a60 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:00.881718 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d5eb5d-804f-450b-ac55-0ec477e3ebd6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:00.886781 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:00.927791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c307d6c-a3b2-4e9a-a68e-9c4e9b933300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:00.928438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c307d6c-a3b2-4e9a-a68e-9c4e9b933300 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:00.928658 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c307d6c-a3b2-4e9a-a68e-9c4e9b933300 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:00.928933 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c307d6c-a3b2-4e9a-a68e-9c4e9b933300 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:00.929648 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c307d6c-a3b2-4e9a-a68e-9c4e9b933300 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:00.929951 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c307d6c-a3b2-4e9a-a68e-9c4e9b933300 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:00.931447 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c307d6c-a3b2-4e9a-a68e-9c4e9b933300 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:00.949347 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 316/621] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:00.951698 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5954c7-6504-47fc-877f-dc861326114d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:00.951991 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5954c7-6504-47fc-877f-dc861326114d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:00.952273 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5954c7-6504-47fc-877f-dc861326114d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:00.953724 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 317/622] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:00.954304 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5954c7-6504-47fc-877f-dc861326114d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:00.956744 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d6df58-40c2-4544-a98f-9dcfbca73a60 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:00.994314 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc4608-c726-4749-8c45-84f5f1c61a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:00.995313 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc4608-c726-4749-8c45-84f5f1c61a8f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:00.995691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc4608-c726-4749-8c45-84f5f1c61a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:00.995984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc4608-c726-4749-8c45-84f5f1c61a8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:01.007767 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc4608-c726-4749-8c45-84f5f1c61a8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:01.009299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc4608-c726-4749-8c45-84f5f1c61a8f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:01.025803 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:01.029439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5acc4608-c726-4749-8c45-84f5f1c61a8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:01.029439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:01.029439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:01.029439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:01.029439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:01.029439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:01.029439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:01.029439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:01.029439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:01.029439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:01.029439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:01.029439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:01.029439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:01.029439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:01.029439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:01.030592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:01.030592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:01.030592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:01.030592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:01.030592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:01.030592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:01.030592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:01.030592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:01.030592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:01.030592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:01.030592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:01.030592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:01.030592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:01.030592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:01.030592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:01.030592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:01.030592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:01.031520 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:01.031520 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:01.031520 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:01.031520 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:01.033721 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 306/623] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:01.035209 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf30c5-6ed2-4940-92ef-13a97f85d044 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:01.035720 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf30c5-6ed2-4940-92ef-13a97f85d044 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:01.036213 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf30c5-6ed2-4940-92ef-13a97f85d044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:01.053030 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9bcf30c5-6ed2-4940-92ef-13a97f85d044 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:01.054560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9bcf30c5-6ed2-4940-92ef-13a97f85d044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:01.077999 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5954c7-6504-47fc-877f-dc861326114d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:01.120394 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a3c99-508b-44d1-821d-0613f193821a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:01.121570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a3c99-508b-44d1-821d-0613f193821a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:01.122061 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a3c99-508b-44d1-821d-0613f193821a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:01.122608 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a3c99-508b-44d1-821d-0613f193821a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:01.129753 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a3c99-508b-44d1-821d-0613f193821a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:01.131507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a3c99-508b-44d1-821d-0613f193821a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:01.144522 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 307/624] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:21:01.148882 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-695a3c99-508b-44d1-821d-0613f193821a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:01.148882 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:01.148882 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:01.148882 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:01.148882 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:01.148882 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:01.148882 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:01.148882 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:01.148882 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:01.148882 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:01.148882 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:01.148882 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:01.148882 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:21:01.148882 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:21:01.148882 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:21:01.149784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:21:01.149784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:01.149784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:01.149784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:01.149784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:01.149784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:01.149784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:01.149784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:01.149784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:01.149784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:01.149784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:01.149784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:01.149784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:01.149784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:01.149784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:01.149784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:01.149784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:01.150755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:21:01.150755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:21:01.150755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:21:01.150755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:21:01.150755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:21:01.150755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:21:01.150755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:01.150755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:01.151789 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf30c5-6ed2-4940-92ef-13a97f85d044 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:01.153591 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 318/625] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:21:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:01.160530 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a3c99-508b-44d1-821d-0613f193821a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:01.203315 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:01.269424 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a3c99-508b-44d1-821d-0613f193821a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:01.291558 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:01.303675 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf30c5-6ed2-4940-92ef-13a97f85d044 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:01.311480 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d2464-6b1c-4ca4-89f7-04b2dba65fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:01.312143 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d2464-6b1c-4ca4-89f7-04b2dba65fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:01.313683 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d2464-6b1c-4ca4-89f7-04b2dba65fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:01.313973 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d2464-6b1c-4ca4-89f7-04b2dba65fc6 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:01.314786 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d2464-6b1c-4ca4-89f7-04b2dba65fc6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:01.315096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d2464-6b1c-4ca4-89f7-04b2dba65fc6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:01.316536 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d2464-6b1c-4ca4-89f7-04b2dba65fc6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:01.334059 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 319/626] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:21:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:01.334948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d2464-6b1c-4ca4-89f7-04b2dba65fc6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:01.354207 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140be0c2-837a-48d4-af5f-c9274f4fbb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:01.354986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140be0c2-837a-48d4-af5f-c9274f4fbb26 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:01.355362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140be0c2-837a-48d4-af5f-c9274f4fbb26 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:01.355926 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140be0c2-837a-48d4-af5f-c9274f4fbb26 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:01.356716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140be0c2-837a-48d4-af5f-c9274f4fbb26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:01.357388 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140be0c2-837a-48d4-af5f-c9274f4fbb26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:01.361626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140be0c2-837a-48d4-af5f-c9274f4fbb26 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:01.382505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 308/627] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:01.383592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140be0c2-837a-48d4-af5f-c9274f4fbb26 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:01.447042 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d2464-6b1c-4ca4-89f7-04b2dba65fc6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:01.483638 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bfb71-8c72-43b3-a912-b65962d3db55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:01.484728 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bfb71-8c72-43b3-a912-b65962d3db55 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:01.485203 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bfb71-8c72-43b3-a912-b65962d3db55 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:01.485478 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bfb71-8c72-43b3-a912-b65962d3db55 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:01.496331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73bfb71-8c72-43b3-a912-b65962d3db55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:21:01.504285 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c73bfb71-8c72-43b3-a912-b65962d3db55 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:21:01.504285 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:01.504285 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:01.504285 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:01.504285 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:01.504285 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:01.504285 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:01.504285 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:01.504285 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:01.504285 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:01.504285 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:01.504285 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:01.504285 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:21:01.504285 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:21:01.504285 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:21:01.504285 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:21:01.505241 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:21:01.505241 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:21:01.505241 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:21:01.505241 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:21:01.505241 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:01.505241 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:01.505241 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:01.505241 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:01.505241 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:01.505241 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:01.505241 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:01.505241 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:01.505241 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:01.505241 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:01.505241 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:01.505241 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:01.506147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:01.506147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:01.506147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:01.506147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:01.506147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:21:01.506147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:21:01.506147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:21:01.506147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:21:01.506147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:21:01.506147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:21:01.506147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:21:01.506147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:01.507103 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:01.510845 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 320/628] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 12:21:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:01.512126 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bfb71-8c72-43b3-a912-b65962d3db55 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:01.521780 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140be0c2-837a-48d4-af5f-c9274f4fbb26 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:01.558322 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6349bb-fc9b-4964-ab94-d262e55a277c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:01.559267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6349bb-fc9b-4964-ab94-d262e55a277c None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:01.559548 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6349bb-fc9b-4964-ab94-d262e55a277c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:01.559824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6349bb-fc9b-4964-ab94-d262e55a277c None admin] PATH_INFO: `/v3/projects/f043ade9e4f748d4adc010ffb0a47a46/users/209b0df45f76404da880a9ae0c637d67/roles/8883c151055d4ba493f514ce1bb62e78` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:01.565986 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:01.576159 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6349bb-fc9b-4964-ab94-d262e55a277c None admin] RBAC: Authorizing `identity:create_grant(project_id=f043ade9e4f748d4adc010ffb0a47a46, user_id=209b0df45f76404da880a9ae0c637d67, role_id=8883c151055d4ba493f514ce1bb62e78)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:01.577600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6349bb-fc9b-4964-ab94-d262e55a277c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:01.592649 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 309/629] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 12:21:01 2022] PUT /identity/v3/projects/f043ade9e4f748d4adc010ffb0a47a46/users/209b0df45f76404da880a9ae0c637d67/roles/8883c151055d4ba493f514ce1bb62e78 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:21:01.593759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6349bb-fc9b-4964-ab94-d262e55a277c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:01.641874 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bfb71-8c72-43b3-a912-b65962d3db55 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:01.690370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344cbdb5-fb2f-4ba9-8e65-2ab13a7a09fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:01.691686 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344cbdb5-fb2f-4ba9-8e65-2ab13a7a09fc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:01.691978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344cbdb5-fb2f-4ba9-8e65-2ab13a7a09fc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:01.692261 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344cbdb5-fb2f-4ba9-8e65-2ab13a7a09fc None admin] PATH_INFO: `/v3/users/glance` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:01.703322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344cbdb5-fb2f-4ba9-8e65-2ab13a7a09fc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:01.705501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344cbdb5-fb2f-4ba9-8e65-2ab13a7a09fc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:01.719117 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6349bb-fc9b-4964-ab94-d262e55a277c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:01.725872 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-344cbdb5-fb2f-4ba9-8e65-2ab13a7a09fc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 29 12:21:01.725872 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:01.725872 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:01.725872 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:01.725872 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:01.725872 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:01.725872 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:01.725872 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:01.725872 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:01.725872 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:01.725872 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:01.725872 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:01.725872 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:21:01.725872 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:21:01.725872 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:21:01.725872 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:21:01.727237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:01.727237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:01.727237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:21:01.727237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:01.727237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:21:01.727237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:01.727237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:01.727237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:01.727237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:01.727237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:01.727237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:01.727237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:01.727237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:01.727237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:01.727237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:01.727237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:01.727237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:01.727237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:01.728180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:01.728180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:01.728180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:21:01.728180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:21:01.728180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:21:01.728180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:21:01.728180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:21:01.728180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:21:01.728180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 29 12:21:01.728180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:01.728180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 321/630] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:21:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:01.734824 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a888d32-e2fa-4357-adb8-499891f484f4 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:01.734824 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a888d32-e2fa-4357-adb8-499891f484f4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:01.734824 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a888d32-e2fa-4357-adb8-499891f484f4 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:01.739222 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 322/631] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:01 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:01.739890 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a888d32-e2fa-4357-adb8-499891f484f4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:01.772033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a91d4b3-48d9-4ed4-b65f-2fdbffd10d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:01.772728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91d4b3-48d9-4ed4-b65f-2fdbffd10d7c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:01.773155 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91d4b3-48d9-4ed4-b65f-2fdbffd10d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:01.773441 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a91d4b3-48d9-4ed4-b65f-2fdbffd10d7c None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:01.774300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a91d4b3-48d9-4ed4-b65f-2fdbffd10d7c None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:01.774700 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a91d4b3-48d9-4ed4-b65f-2fdbffd10d7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:01.776376 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a91d4b3-48d9-4ed4-b65f-2fdbffd10d7c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:01.786469 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 310/632] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 12:21:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:01.787955 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825565c2-9f1e-4f87-a944-757ce45d6f22 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:01.788256 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825565c2-9f1e-4f87-a944-757ce45d6f22 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:01.788538 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825565c2-9f1e-4f87-a944-757ce45d6f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:01.794368 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:01.810768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-825565c2-9f1e-4f87-a944-757ce45d6f22 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:01.812268 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-825565c2-9f1e-4f87-a944-757ce45d6f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:01.883743 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:01.888958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a888d32-e2fa-4357-adb8-499891f484f4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:01.894447 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 311/633] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:21:01.894447 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825565c2-9f1e-4f87-a944-757ce45d6f22 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:01.945583 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a997b-6ac9-4ccd-93f1-de0ead3ef1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:01.946262 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a997b-6ac9-4ccd-93f1-de0ead3ef1a3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:01.946584 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a997b-6ac9-4ccd-93f1-de0ead3ef1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:01.946867 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a997b-6ac9-4ccd-93f1-de0ead3ef1a3 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:01.948526 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a997b-6ac9-4ccd-93f1-de0ead3ef1a3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:01.948813 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a997b-6ac9-4ccd-93f1-de0ead3ef1a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:01.950467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a997b-6ac9-4ccd-93f1-de0ead3ef1a3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:01.974676 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 323/634] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Jun 29 12:21:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:01.974676 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a997b-6ac9-4ccd-93f1-de0ead3ef1a3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:02.026758 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825565c2-9f1e-4f87-a944-757ce45d6f22 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:02.058051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:02.065470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d907f-f728-48c0-adab-f269200fd9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:02.066678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d907f-f728-48c0-adab-f269200fd9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:02.067118 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d907f-f728-48c0-adab-f269200fd9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:02.067572 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3d907f-f728-48c0-adab-f269200fd9a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:02.079790 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d907f-f728-48c0-adab-f269200fd9a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:02.081575 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3d907f-f728-48c0-adab-f269200fd9a1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:02.099738 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a997b-6ac9-4ccd-93f1-de0ead3ef1a3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:02.106273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ca3d907f-f728-48c0-adab-f269200fd9a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:02.106273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:02.106273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:02.106273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:02.106273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:02.106273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:02.106273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:02.106273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:02.106273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:02.106273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:02.106273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:02.106273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:02.106273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:02.106273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:02.106273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:02.107464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:02.107464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:02.107464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:02.107464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:02.107464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:02.107464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:02.107464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:02.107464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:02.107464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:02.107464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:02.107464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:02.107464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:02.107464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:02.107464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:02.107464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:02.107464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:02.107464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:02.108370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:02.108370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:02.108370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:02.108370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:02.108370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 312/635] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:02.109410 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d907f-f728-48c0-adab-f269200fd9a1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:02.148318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37468f4-ba8d-452f-8421-703848ff75f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:02.149109 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37468f4-ba8d-452f-8421-703848ff75f0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:02.149548 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37468f4-ba8d-452f-8421-703848ff75f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:02.149832 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37468f4-ba8d-452f-8421-703848ff75f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:02.163907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37468f4-ba8d-452f-8421-703848ff75f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:21:02.173232 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a37468f4-ba8d-452f-8421-703848ff75f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:21:02.173232 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:02.173232 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:02.173232 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:02.173232 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:02.173232 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:02.173232 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:02.173232 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:02.173232 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:02.173232 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:02.173232 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:02.173232 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:02.173232 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:21:02.173232 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:21:02.173232 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:21:02.173232 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:21:02.174189 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:21:02.174189 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:21:02.174189 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:21:02.174189 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:21:02.174189 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:02.174189 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:02.174189 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:02.174189 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:02.174189 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:02.174189 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:02.174189 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:02.174189 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:02.174189 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:02.174189 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:02.174189 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:02.174189 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:02.175084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:02.175084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:02.175084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:02.175084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:02.175084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:21:02.175084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:21:02.175084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:21:02.175084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:21:02.175084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:21:02.175084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:21:02.175084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:21:02.175084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:02.175084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 324/636] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 12:21:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:02.175825 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37468f4-ba8d-452f-8421-703848ff75f0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:02.199592 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:02.253388 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3d907f-f728-48c0-adab-f269200fd9a1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:02.291205 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb185a4a-7ff3-4640-a4fa-5b9533658583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:02.291789 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb185a4a-7ff3-4640-a4fa-5b9533658583 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:02.292080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb185a4a-7ff3-4640-a4fa-5b9533658583 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:02.292363 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb185a4a-7ff3-4640-a4fa-5b9533658583 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:02.295914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37468f4-ba8d-452f-8421-703848ff75f0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:02.330445 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27737c-11dc-49db-bae8-5e5b702d50ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:02.331130 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27737c-11dc-49db-bae8-5e5b702d50ec None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:02.331420 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27737c-11dc-49db-bae8-5e5b702d50ec None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:02.331702 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27737c-11dc-49db-bae8-5e5b702d50ec None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:02.332435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27737c-11dc-49db-bae8-5e5b702d50ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:02.332752 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27737c-11dc-49db-bae8-5e5b702d50ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:02.334133 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27737c-11dc-49db-bae8-5e5b702d50ec None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:02.346883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 325/637] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:02.347792 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27737c-11dc-49db-bae8-5e5b702d50ec None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:02.356387 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:02.376069 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb185a4a-7ff3-4640-a4fa-5b9533658583 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:02.377548 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb185a4a-7ff3-4640-a4fa-5b9533658583 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:02.436496 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:02.459123 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27737c-11dc-49db-bae8-5e5b702d50ec None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:02.490160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 313/638] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:21:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:21:02.491812 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d06276-219a-4f45-9b1c-6c387586dcb2 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:02.492269 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d06276-219a-4f45-9b1c-6c387586dcb2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:02.492687 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d06276-219a-4f45-9b1c-6c387586dcb2 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:02.493116 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fb3d7-c143-465e-95ae-e11d61ff0a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:02.493785 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fb3d7-c143-465e-95ae-e11d61ff0a40 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:02.494114 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fb3d7-c143-465e-95ae-e11d61ff0a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:02.495519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fb3d7-c143-465e-95ae-e11d61ff0a40 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:02.495626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 314/639] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:02.496378 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fb3d7-c143-465e-95ae-e11d61ff0a40 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:02.496587 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d06276-219a-4f45-9b1c-6c387586dcb2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:02.496682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fb3d7-c143-465e-95ae-e11d61ff0a40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:02.498077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fb3d7-c143-465e-95ae-e11d61ff0a40 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:02.506564 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 326/640] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 12:21:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:02.507361 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fb3d7-c143-465e-95ae-e11d61ff0a40 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:02.595818 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:02.615006 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d06276-219a-4f45-9b1c-6c387586dcb2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:02.616758 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fb3d7-c143-465e-95ae-e11d61ff0a40 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:02.652585 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf977f4-f13b-4b79-906a-ec3e274b2145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:02.653619 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf977f4-f13b-4b79-906a-ec3e274b2145 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:02.653783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e62b2-ec84-49ac-bdd3-6be260af7c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:02.653987 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf977f4-f13b-4b79-906a-ec3e274b2145 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:02.654182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf977f4-f13b-4b79-906a-ec3e274b2145 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:02.654508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e62b2-ec84-49ac-bdd3-6be260af7c7c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:02.654794 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e62b2-ec84-49ac-bdd3-6be260af7c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:02.655072 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e62b2-ec84-49ac-bdd3-6be260af7c7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:02.661576 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf977f4-f13b-4b79-906a-ec3e274b2145 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:02.663042 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf977f4-f13b-4b79-906a-ec3e274b2145 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:02.665753 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:02.669110 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e62b2-ec84-49ac-bdd3-6be260af7c7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:02.670709 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e62b2-ec84-49ac-bdd3-6be260af7c7c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:02.679436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-caf977f4-f13b-4b79-906a-ec3e274b2145 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 12:21:02.679436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:02.679436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:02.679436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:02.679436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:02.679436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:02.679436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:02.679436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:02.679436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:02.679436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:02.679436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:02.679436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:02.679436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:21:02.679436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:21:02.679436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:21:02.679436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:21:02.680372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:02.680372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:02.680372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:21:02.680372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:02.680372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:21:02.680372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:02.680372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:02.680372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:02.680372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:02.680372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:02.680372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:02.680372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:02.680372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:02.680372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:02.680372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:02.680372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:02.680372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:02.680372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:02.681310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:02.681310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:02.681310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:21:02.681310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:21:02.681310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:21:02.681310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:21:02.681310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:21:02.681310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:21:02.681310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 12:21:02.681310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:02.681310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 315/641] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Jun 29 12:21:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:02.684121 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae750b-110d-4260-afa6-280d82c65c93 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:02.684121 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae750b-110d-4260-afa6-280d82c65c93 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:02.684121 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae750b-110d-4260-afa6-280d82c65c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:02.696478 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a05e62b2-ec84-49ac-bdd3-6be260af7c7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:02.696478 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:02.696478 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:02.696478 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:02.696478 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:02.696478 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:02.696478 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:02.696478 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:02.696478 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:02.696478 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:02.696478 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:02.696478 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:02.696478 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:02.696478 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:02.696478 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:02.697653 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:02.697653 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:02.697653 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:02.697653 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:02.697653 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:02.697653 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:02.697653 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:02.697653 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:02.697653 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:02.697653 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:02.697653 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:02.697653 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:02.697653 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:02.697653 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:02.697653 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:02.697653 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:02.697653 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:02.698731 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:02.698731 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:02.698731 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:02.698731 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:02.698731 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 327/642] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:02.699031 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e62b2-ec84-49ac-bdd3-6be260af7c7c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:02.705643 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-36ae750b-110d-4260-afa6-280d82c65c93 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:02.706900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-36ae750b-110d-4260-afa6-280d82c65c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:02.778053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 316/643] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:21:02.779670 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ae750b-110d-4260-afa6-280d82c65c93 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:02.822406 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e62b2-ec84-49ac-bdd3-6be260af7c7c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:02.860294 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:02.865408 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e4534-ce75-41c0-a175-5a59eeed2a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:02.866196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e4534-ce75-41c0-a175-5a59eeed2a71 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:02.866506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e4534-ce75-41c0-a175-5a59eeed2a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:02.866785 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e4534-ce75-41c0-a175-5a59eeed2a71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:02.877924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288e4534-ce75-41c0-a175-5a59eeed2a71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:02.879435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288e4534-ce75-41c0-a175-5a59eeed2a71 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:02.888678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ae750b-110d-4260-afa6-280d82c65c93 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:02.897481 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-288e4534-ce75-41c0-a175-5a59eeed2a71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:02.897481 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:02.897481 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:02.897481 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:02.897481 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:02.897481 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:02.897481 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:02.897481 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:02.897481 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:02.897481 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:02.897481 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:02.897481 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:02.897481 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:02.897481 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:02.897481 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:02.898382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:02.898382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:02.898382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:02.898382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:02.898382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:02.898382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:02.898382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:02.898382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:02.898382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:02.898382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:02.898382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:02.898382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:02.898382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:02.898382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:02.898382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:02.898382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:02.898382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:02.899537 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:02.899537 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:02.899537 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:02.899537 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:02.899537 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 328/644] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:02.899822 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e4534-ce75-41c0-a175-5a59eeed2a71 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:02.928150 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b861d1e-90bb-4486-8d5f-7fa212101a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:02.928837 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b861d1e-90bb-4486-8d5f-7fa212101a7c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:02.929129 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b861d1e-90bb-4486-8d5f-7fa212101a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:02.929411 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b861d1e-90bb-4486-8d5f-7fa212101a7c None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:02.930248 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b861d1e-90bb-4486-8d5f-7fa212101a7c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:02.930595 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b861d1e-90bb-4486-8d5f-7fa212101a7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:02.932617 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b861d1e-90bb-4486-8d5f-7fa212101a7c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:02.948782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:02.949708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 317/645] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Jun 29 12:21:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:02.950545 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b861d1e-90bb-4486-8d5f-7fa212101a7c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:03.010023 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e4534-ce75-41c0-a175-5a59eeed2a71 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:03.044604 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739d890-6409-48b6-bf97-0b4d9711a666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:03.045430 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739d890-6409-48b6-bf97-0b4d9711a666 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:03.045727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739d890-6409-48b6-bf97-0b4d9711a666 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:03.045995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4739d890-6409-48b6-bf97-0b4d9711a666 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:03.046746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739d890-6409-48b6-bf97-0b4d9711a666 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:03.047067 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739d890-6409-48b6-bf97-0b4d9711a666 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:03.048667 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4739d890-6409-48b6-bf97-0b4d9711a666 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:03.050927 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b861d1e-90bb-4486-8d5f-7fa212101a7c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:03.065283 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 329/646] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:03.066354 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739d890-6409-48b6-bf97-0b4d9711a666 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:03.080931 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:03.086510 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad530ada-2090-4804-ba7c-1cab401593b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:03.087495 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad530ada-2090-4804-ba7c-1cab401593b8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:03.088024 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad530ada-2090-4804-ba7c-1cab401593b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:03.088473 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad530ada-2090-4804-ba7c-1cab401593b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:03.100126 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad530ada-2090-4804-ba7c-1cab401593b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:21:03.108962 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ad530ada-2090-4804-ba7c-1cab401593b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:21:03.108962 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:03.108962 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:03.108962 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:03.108962 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:03.108962 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:03.108962 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:03.108962 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:03.108962 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:03.108962 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:03.108962 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:03.108962 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:03.108962 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:21:03.108962 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:21:03.108962 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:21:03.108962 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:21:03.109923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:21:03.109923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:21:03.109923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:21:03.109923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:21:03.109923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:03.109923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:03.109923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:03.109923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:03.109923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:03.109923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:03.109923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:03.109923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:03.109923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:03.109923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:03.109923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:03.109923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:03.110856 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:03.110856 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:03.110856 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:03.110856 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:03.110856 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:21:03.110856 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:21:03.110856 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:21:03.110856 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:21:03.110856 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:21:03.110856 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:21:03.110856 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:21:03.110856 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:03.111513 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 318/647] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 12:21:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:03.112152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad530ada-2090-4804-ba7c-1cab401593b8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:03.176888 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4739d890-6409-48b6-bf97-0b4d9711a666 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:03.178211 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:03.216344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d8199b-8ac6-4086-bc43-95cd71daabcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:03.217525 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8199b-8ac6-4086-bc43-95cd71daabcc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:03.217813 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8199b-8ac6-4086-bc43-95cd71daabcc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:03.218095 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d8199b-8ac6-4086-bc43-95cd71daabcc None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:03.219126 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d8199b-8ac6-4086-bc43-95cd71daabcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:03.219439 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d8199b-8ac6-4086-bc43-95cd71daabcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:03.221711 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d8199b-8ac6-4086-bc43-95cd71daabcc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:03.230492 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad530ada-2090-4804-ba7c-1cab401593b8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:03.241417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 330/648] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:03.241417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d8199b-8ac6-4086-bc43-95cd71daabcc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:03.267581 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f02fe-d9ce-433f-877a-b073c10731ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:03.268234 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f02fe-d9ce-433f-877a-b073c10731ee None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:03.268524 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f02fe-d9ce-433f-877a-b073c10731ee None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:03.268806 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f02fe-d9ce-433f-877a-b073c10731ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:03.335576 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:03.350038 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f02fe-d9ce-433f-877a-b073c10731ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:03.351975 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76f02fe-d9ce-433f-877a-b073c10731ee None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:03.355307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d8199b-8ac6-4086-bc43-95cd71daabcc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:03.391946 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb95b7-8d44-4004-82b9-632073914d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:03.392744 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb95b7-8d44-4004-82b9-632073914d6a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:03.393030 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb95b7-8d44-4004-82b9-632073914d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:03.393306 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb95b7-8d44-4004-82b9-632073914d6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:03.401068 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb95b7-8d44-4004-82b9-632073914d6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:03.402687 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb95b7-8d44-4004-82b9-632073914d6a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:03.417687 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-18fb95b7-8d44-4004-82b9-632073914d6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:03.417687 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:03.417687 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:03.417687 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:03.417687 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:03.417687 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:03.417687 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:03.417687 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:03.417687 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:03.417687 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:03.417687 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:03.417687 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:03.417687 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:21:03.417687 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:21:03.417687 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:21:03.418907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:21:03.418907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:03.418907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:03.418907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:03.418907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:03.418907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:03.418907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:03.418907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:03.418907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:03.418907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:03.418907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:03.418907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:03.418907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:03.418907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:03.418907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:03.418907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:03.418907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:03.419791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:21:03.419791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:21:03.419791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:21:03.419791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:21:03.419791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:21:03.419791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:21:03.419791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:03.419791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:03.419791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 331/649] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:21:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:03.419791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb95b7-8d44-4004-82b9-632073914d6a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:03.446785 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:03.466719 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 319/650] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:21:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:21:03.471551 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f02fe-d9ce-433f-877a-b073c10731ee None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:03.535123 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb95b7-8d44-4004-82b9-632073914d6a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:03.569419 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658ef59-364f-4bfe-8267-7389b634200b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:03.570095 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658ef59-364f-4bfe-8267-7389b634200b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:03.570381 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658ef59-364f-4bfe-8267-7389b634200b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:03.570689 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658ef59-364f-4bfe-8267-7389b634200b None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:03.571439 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658ef59-364f-4bfe-8267-7389b634200b None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:03.571750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658ef59-364f-4bfe-8267-7389b634200b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:03.573836 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658ef59-364f-4bfe-8267-7389b634200b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:03.575976 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:03.587316 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 332/651] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 12:21:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:03.587316 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658ef59-364f-4bfe-8267-7389b634200b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:03.592007 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f02fe-d9ce-433f-877a-b073c10731ee None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:03.628999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e4fec-715a-48fb-a59e-dd5cff480579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:03.629830 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e4fec-715a-48fb-a59e-dd5cff480579 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:03.630109 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e4fec-715a-48fb-a59e-dd5cff480579 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:03.630392 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e4fec-715a-48fb-a59e-dd5cff480579 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:03.639415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706e4fec-715a-48fb-a59e-dd5cff480579 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:03.640992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706e4fec-715a-48fb-a59e-dd5cff480579 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:03.657385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-706e4fec-715a-48fb-a59e-dd5cff480579 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 12:21:03.657385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:03.657385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:03.657385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:03.657385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:03.657385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:03.657385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:03.657385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:03.657385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:03.657385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:03.657385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:03.657385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:03.657385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:21:03.657385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:21:03.657385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:21:03.657385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:21:03.658331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:03.658331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:03.658331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:21:03.658331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:03.658331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:21:03.658331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:03.658331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:03.658331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:03.658331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:03.658331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:03.658331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:03.658331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:03.658331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:03.658331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:03.658331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:03.658331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:03.658331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:03.658331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:03.659317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:03.659317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:03.659317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:21:03.659317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:21:03.659317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:21:03.659317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:21:03.659317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:21:03.659317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:21:03.659317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 12:21:03.659317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:03.659317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 320/652] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:21:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:03.659908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e4fec-715a-48fb-a59e-dd5cff480579 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:03.676991 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:03.694456 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658ef59-364f-4bfe-8267-7389b634200b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:03.731590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02188e2d-b8a1-4adf-9817-16db04b2ae2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:03.732257 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02188e2d-b8a1-4adf-9817-16db04b2ae2e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:03.732538 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02188e2d-b8a1-4adf-9817-16db04b2ae2e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:03.732816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02188e2d-b8a1-4adf-9817-16db04b2ae2e None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:03.733600 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02188e2d-b8a1-4adf-9817-16db04b2ae2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:03.733910 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02188e2d-b8a1-4adf-9817-16db04b2ae2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:03.735363 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02188e2d-b8a1-4adf-9817-16db04b2ae2e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:03.747967 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 333/653] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:21:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:03.748778 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02188e2d-b8a1-4adf-9817-16db04b2ae2e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:03.765455 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e4fec-715a-48fb-a59e-dd5cff480579 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:03.798821 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef226f-eb2a-490a-8c6f-9b5bc4c99eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:03.799604 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef226f-eb2a-490a-8c6f-9b5bc4c99eaa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:03.799886 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef226f-eb2a-490a-8c6f-9b5bc4c99eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:03.800161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef226f-eb2a-490a-8c6f-9b5bc4c99eaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:03.810388 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef226f-eb2a-490a-8c6f-9b5bc4c99eaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:03.811850 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef226f-eb2a-490a-8c6f-9b5bc4c99eaa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:03.830218 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ddef226f-eb2a-490a-8c6f-9b5bc4c99eaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:03.830218 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:03.830218 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:03.830218 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:03.830218 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:03.830218 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:03.830218 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:03.830218 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:03.830218 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:03.830218 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:03.830218 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:03.830218 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:03.830218 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:03.830218 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:03.830218 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:03.831160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:03.831160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:03.831160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:03.831160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:03.831160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:03.831160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:03.831160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:03.831160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:03.831160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:03.831160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:03.831160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:03.831160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:03.831160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:03.831160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:03.831160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:03.831160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:03.831160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:03.832071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:03.832071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:03.832071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:03.832071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:03.832071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 321/654] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:03.832370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef226f-eb2a-490a-8c6f-9b5bc4c99eaa None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:03.835648 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:03.854234 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02188e2d-b8a1-4adf-9817-16db04b2ae2e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:03.887747 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d7c112-8d24-4cab-a275-5b82552afeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:03.888463 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7c112-8d24-4cab-a275-5b82552afeb9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:03.888750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7c112-8d24-4cab-a275-5b82552afeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:03.889032 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d7c112-8d24-4cab-a275-5b82552afeb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:03.899336 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d7c112-8d24-4cab-a275-5b82552afeb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:03.901387 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d7c112-8d24-4cab-a275-5b82552afeb9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:03.907959 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:03.919550 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c3d7c112-8d24-4cab-a275-5b82552afeb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:03.919550 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:03.919550 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:03.919550 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:03.919550 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:03.919550 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:03.919550 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:03.919550 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:03.919550 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:03.919550 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:03.919550 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:03.919550 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:03.919550 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:03.919550 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:03.919550 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:03.920438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:03.920438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:03.920438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:03.920438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:03.920438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:03.920438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:03.920438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:03.920438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:03.920438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:03.920438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:03.920438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:03.920438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:03.920438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:03.920438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:03.920438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:03.920438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:03.920438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:03.921358 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:03.921358 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:03.921358 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:03.921358 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:03.921358 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 334/655] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:03.922009 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d520c0-8db9-46f4-9031-8aec238f07be None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:03.922250 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d520c0-8db9-46f4-9031-8aec238f07be None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:03.922557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d520c0-8db9-46f4-9031-8aec238f07be None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:03.923807 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 335/656] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:03.924500 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d520c0-8db9-46f4-9031-8aec238f07be None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:03.936181 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef226f-eb2a-490a-8c6f-9b5bc4c99eaa None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:03.970179 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd041ee-01b9-4a96-8e7d-8980901f3a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:03.970875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd041ee-01b9-4a96-8e7d-8980901f3a01 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:03.971156 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd041ee-01b9-4a96-8e7d-8980901f3a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:03.971436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd041ee-01b9-4a96-8e7d-8980901f3a01 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:03.972292 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd041ee-01b9-4a96-8e7d-8980901f3a01 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:03.972596 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd041ee-01b9-4a96-8e7d-8980901f3a01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:03.974027 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd041ee-01b9-4a96-8e7d-8980901f3a01 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:03.990709 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 322/657] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Jun 29 12:21:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:03.991508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd041ee-01b9-4a96-8e7d-8980901f3a01 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:04.024162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d520c0-8db9-46f4-9031-8aec238f07be None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:04.053977 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:04.058347 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a9add-55c3-49a3-a1c1-c5dafad134cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:04.059064 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a9add-55c3-49a3-a1c1-c5dafad134cb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:04.059354 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a9add-55c3-49a3-a1c1-c5dafad134cb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:04.059638 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a9add-55c3-49a3-a1c1-c5dafad134cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:04.060471 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a9add-55c3-49a3-a1c1-c5dafad134cb None admin] RBAC: Adding query filter params (role.id=8883c151055d4ba493f514ce1bb62e78, scope.project.id=f043ade9e4f748d4adc010ffb0a47a46, user.id=096234e9f15541c0b187dc8dd527b0fa) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:04.060794 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a9add-55c3-49a3-a1c1-c5dafad134cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:04.062237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a9add-55c3-49a3-a1c1-c5dafad134cb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:04.068263 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 336/658] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:21:03 2022] GET /identity/v3/role_assignments?user.id=096234e9f15541c0b187dc8dd527b0fa&scope.project.id=f043ade9e4f748d4adc010ffb0a47a46&role.id=8883c151055d4ba493f514ce1bb62e78 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:04.068979 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a9add-55c3-49a3-a1c1-c5dafad134cb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:04.096735 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd041ee-01b9-4a96-8e7d-8980901f3a01 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:04.137964 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b42f2-8dd3-446e-b849-de64035a9584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:04.138619 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b42f2-8dd3-446e-b849-de64035a9584 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:04.138756 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:04.138928 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b42f2-8dd3-446e-b849-de64035a9584 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:04.139194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b42f2-8dd3-446e-b849-de64035a9584 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:04.140028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b42f2-8dd3-446e-b849-de64035a9584 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:04.140415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b42f2-8dd3-446e-b849-de64035a9584 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:04.141825 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b42f2-8dd3-446e-b849-de64035a9584 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:04.160800 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 323/659] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:04.160800 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df852da-6477-45a9-9874-89a6cabba4a8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:04.160800 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df852da-6477-45a9-9874-89a6cabba4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:04.160800 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df852da-6477-45a9-9874-89a6cabba4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:04.177467 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9df852da-6477-45a9-9874-89a6cabba4a8 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:04.178839 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9df852da-6477-45a9-9874-89a6cabba4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:04.183848 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a9add-55c3-49a3-a1c1-c5dafad134cb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:04.224772 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa95e50-8859-4911-8d3c-a1b133a7189e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:04.225299 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa95e50-8859-4911-8d3c-a1b133a7189e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:04.225593 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa95e50-8859-4911-8d3c-a1b133a7189e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:04.225864 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa95e50-8859-4911-8d3c-a1b133a7189e None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:04.226623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa95e50-8859-4911-8d3c-a1b133a7189e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:04.226940 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa95e50-8859-4911-8d3c-a1b133a7189e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:04.229160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa95e50-8859-4911-8d3c-a1b133a7189e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:04.242028 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 337/660] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:04.243255 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa95e50-8859-4911-8d3c-a1b133a7189e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:04.263025 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 324/661] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:21:04.270455 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df852da-6477-45a9-9874-89a6cabba4a8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:04.321170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:04.365043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa95e50-8859-4911-8d3c-a1b133a7189e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:04.373957 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df852da-6477-45a9-9874-89a6cabba4a8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:04.390079 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:04.403260 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca375a4-de93-4649-83f1-6a2e31b22c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:04.403675 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca375a4-de93-4649-83f1-6a2e31b22c05 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:04.403958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca375a4-de93-4649-83f1-6a2e31b22c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:04.404257 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca375a4-de93-4649-83f1-6a2e31b22c05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:04.411155 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608da8d-2144-469b-a497-5f06431347ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:04.411960 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608da8d-2144-469b-a497-5f06431347ee None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:04.412245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608da8d-2144-469b-a497-5f06431347ee None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:04.412522 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608da8d-2144-469b-a497-5f06431347ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:04.420047 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4608da8d-2144-469b-a497-5f06431347ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:04.421497 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4608da8d-2144-469b-a497-5f06431347ee None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:04.435470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4608da8d-2144-469b-a497-5f06431347ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:04.435470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:04.435470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:04.435470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:04.435470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:04.435470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:04.435470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:04.435470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:04.435470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:04.435470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:04.435470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:04.435470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:04.435470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:21:04.435470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:21:04.435470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:21:04.436342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:21:04.436342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:04.436342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:04.436342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:04.436342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:04.436342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:04.436342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:04.436342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:04.436342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:04.436342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:04.436342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:04.436342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:04.436342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:04.436342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:04.436342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:04.436342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:04.436342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:04.437228 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:21:04.437228 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:21:04.437228 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:21:04.437228 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:21:04.437228 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:21:04.437228 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:21:04.437228 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:04.437228 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:04.437228 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 325/662] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:21:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:04.437708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608da8d-2144-469b-a497-5f06431347ee None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:04.486805 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca375a4-de93-4649-83f1-6a2e31b22c05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:04.488281 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca375a4-de93-4649-83f1-6a2e31b22c05 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:04.558286 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608da8d-2144-469b-a497-5f06431347ee None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:04.572676 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:04.597234 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 338/663] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:21:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:21:04.597234 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca375a4-de93-4649-83f1-6a2e31b22c05 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:04.597485 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34827b0f-f20a-4b62-b6ed-964dde23f67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:04.597485 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34827b0f-f20a-4b62-b6ed-964dde23f67d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:04.597812 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34827b0f-f20a-4b62-b6ed-964dde23f67d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:04.597986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34827b0f-f20a-4b62-b6ed-964dde23f67d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:04.607151 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34827b0f-f20a-4b62-b6ed-964dde23f67d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:04.608600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34827b0f-f20a-4b62-b6ed-964dde23f67d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:04.623827 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:04.624224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-34827b0f-f20a-4b62-b6ed-964dde23f67d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 12:21:04.624224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:04.624224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:04.624224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:04.624224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:04.624224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:04.624224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:04.624224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:04.624224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:04.624224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:04.624224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:04.624224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:04.624224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:21:04.624224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:21:04.624224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:21:04.624224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:21:04.625210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:04.625210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:04.625210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:21:04.625210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:04.625210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:21:04.625210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:04.625210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:04.625210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:04.625210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:04.625210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:04.625210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:04.625210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:04.625210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:04.625210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:04.625210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:04.625210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:04.625210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:04.625210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:04.626161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:04.626161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:04.626161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:21:04.626161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:21:04.626161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:21:04.626161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:21:04.626161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:21:04.626161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:21:04.626161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 29 12:21:04.626161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:04.626161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 326/664] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 12:21:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:04.626812 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34827b0f-f20a-4b62-b6ed-964dde23f67d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:04.705360 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca375a4-de93-4649-83f1-6a2e31b22c05 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:04.735400 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34827b0f-f20a-4b62-b6ed-964dde23f67d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:04.742285 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135377a-a6a3-40eb-b81a-e18a8ab9599e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:04.743109 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135377a-a6a3-40eb-b81a-e18a8ab9599e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:04.743392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135377a-a6a3-40eb-b81a-e18a8ab9599e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:04.743665 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135377a-a6a3-40eb-b81a-e18a8ab9599e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:04.754404 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135377a-a6a3-40eb-b81a-e18a8ab9599e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:21:04.762508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e135377a-a6a3-40eb-b81a-e18a8ab9599e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:21:04.762508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:04.762508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:04.762508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:04.762508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:04.762508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:04.762508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:04.762508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:04.762508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:04.762508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:04.762508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:04.762508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:04.762508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:21:04.762508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:21:04.762508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:21:04.762508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:21:04.763457 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:21:04.763457 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:21:04.763457 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:21:04.763457 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:21:04.763457 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:04.763457 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:04.763457 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:04.763457 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:04.763457 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:04.763457 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:04.763457 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:04.763457 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:04.763457 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:04.763457 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:04.763457 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:04.763457 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:04.764328 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:04.764328 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:04.764328 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:04.764328 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:04.764328 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:21:04.764328 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:21:04.764328 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:21:04.764328 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:21:04.764328 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:21:04.764328 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:21:04.764328 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:21:04.764328 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:04.764328 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 339/665] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 12:21:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:04.765041 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135377a-a6a3-40eb-b81a-e18a8ab9599e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:04.776999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485e360-c159-4687-a507-cc4bbf89ca13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:04.777664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485e360-c159-4687-a507-cc4bbf89ca13 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:04.777947 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485e360-c159-4687-a507-cc4bbf89ca13 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:04.778221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485e360-c159-4687-a507-cc4bbf89ca13 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:04.779274 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2485e360-c159-4687-a507-cc4bbf89ca13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:04.779592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2485e360-c159-4687-a507-cc4bbf89ca13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:04.781020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2485e360-c159-4687-a507-cc4bbf89ca13 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:04.794597 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 327/666] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:21:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:04.795499 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485e360-c159-4687-a507-cc4bbf89ca13 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:04.832996 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:04.857626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:04.876109 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135377a-a6a3-40eb-b81a-e18a8ab9599e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:04.910204 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485e360-c159-4687-a507-cc4bbf89ca13 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:04.913575 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250fb06a-4d67-48af-a9a9-6ad1b4cb97de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:04.914364 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fb06a-4d67-48af-a9a9-6ad1b4cb97de None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:04.914678 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fb06a-4d67-48af-a9a9-6ad1b4cb97de None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:04.914958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250fb06a-4d67-48af-a9a9-6ad1b4cb97de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:04.926582 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250fb06a-4d67-48af-a9a9-6ad1b4cb97de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:04.928027 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250fb06a-4d67-48af-a9a9-6ad1b4cb97de None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:04.945111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-250fb06a-4d67-48af-a9a9-6ad1b4cb97de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:04.945111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:04.945111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:04.945111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:04.945111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:04.945111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:04.945111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:04.945111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:04.945111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:04.945111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:04.945111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:04.945111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:04.945111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:04.945111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:04.945111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:04.946054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:04.946054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:04.946054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:04.946054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:04.946054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:04.946054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:04.946054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:04.946054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:04.946054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:04.946054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:04.946054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:04.946054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:04.946054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:04.946054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:04.946054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:04.946054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:04.946054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:04.947796 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:04.947796 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:04.947796 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:04.947796 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:04.947796 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 340/667] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:04.947796 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250fb06a-4d67-48af-a9a9-6ad1b4cb97de None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:04.952421 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f604425-d928-436d-a4b6-f7364aff4b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:04.953103 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f604425-d928-436d-a4b6-f7364aff4b22 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:04.953395 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f604425-d928-436d-a4b6-f7364aff4b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:04.953737 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f604425-d928-436d-a4b6-f7364aff4b22 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:04.954615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f604425-d928-436d-a4b6-f7364aff4b22 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:04.954932 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f604425-d928-436d-a4b6-f7364aff4b22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:04.956410 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f604425-d928-436d-a4b6-f7364aff4b22 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:04.974876 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 328/668] 10.222.0.52 () {56 vars in 1139 bytes} [Wed Jun 29 12:21:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:04.980487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f604425-d928-436d-a4b6-f7364aff4b22 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:05.049576 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250fb06a-4d67-48af-a9a9-6ad1b4cb97de None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:05.076840 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:05.086805 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93433c81-8d73-436d-a282-c7e252c3f57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:05.087477 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93433c81-8d73-436d-a282-c7e252c3f57d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:05.088287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93433c81-8d73-436d-a282-c7e252c3f57d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:05.089033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f604425-d928-436d-a4b6-f7364aff4b22 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:05.089150 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93433c81-8d73-436d-a282-c7e252c3f57d None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:05.089792 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93433c81-8d73-436d-a282-c7e252c3f57d None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:05.090102 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93433c81-8d73-436d-a282-c7e252c3f57d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:05.091927 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93433c81-8d73-436d-a282-c7e252c3f57d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:05.092967 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:05.105228 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 341/669] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 12:21:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:05.105228 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93433c81-8d73-436d-a282-c7e252c3f57d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:05.129081 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66baf6-5463-4344-a53a-82d986b4d994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:05.129744 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66baf6-5463-4344-a53a-82d986b4d994 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:05.130056 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66baf6-5463-4344-a53a-82d986b4d994 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:05.130332 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66baf6-5463-4344-a53a-82d986b4d994 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:05.131198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca66baf6-5463-4344-a53a-82d986b4d994 None admin] RBAC: Adding query filter params (role.id=8883c151055d4ba493f514ce1bb62e78, scope.project.id=f043ade9e4f748d4adc010ffb0a47a46, user.id=3ac7391a3c41499d94e5f97478797313) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:05.131509 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca66baf6-5463-4344-a53a-82d986b4d994 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:05.132926 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca66baf6-5463-4344-a53a-82d986b4d994 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:05.139190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 329/670] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:21:04 2022] GET /identity/v3/role_assignments?user.id=3ac7391a3c41499d94e5f97478797313&scope.project.id=f043ade9e4f748d4adc010ffb0a47a46&role.id=8883c151055d4ba493f514ce1bb62e78 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:05.139970 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66baf6-5463-4344-a53a-82d986b4d994 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:05.234003 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93433c81-8d73-436d-a282-c7e252c3f57d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:05.283717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c636543-d261-4830-9e02-3197c2893959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:05.285120 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c636543-d261-4830-9e02-3197c2893959 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:05.285410 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c636543-d261-4830-9e02-3197c2893959 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:05.285689 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c636543-d261-4830-9e02-3197c2893959 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:05.286454 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c636543-d261-4830-9e02-3197c2893959 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:05.286763 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c636543-d261-4830-9e02-3197c2893959 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:05.288018 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66baf6-5463-4344-a53a-82d986b4d994 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:05.288135 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c636543-d261-4830-9e02-3197c2893959 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:05.305828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 342/671] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:05.306795 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c636543-d261-4830-9e02-3197c2893959 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:05.332782 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0e34c-471f-4040-94ed-5e3f57c7e2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:05.333808 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0e34c-471f-4040-94ed-5e3f57c7e2de None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:05.334305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0e34c-471f-4040-94ed-5e3f57c7e2de None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:05.334844 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0e34c-471f-4040-94ed-5e3f57c7e2de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:05.371288 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:05.388420 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:05.428933 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb0e34c-471f-4040-94ed-5e3f57c7e2de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:05.429922 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb0e34c-471f-4040-94ed-5e3f57c7e2de None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:05.446975 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c636543-d261-4830-9e02-3197c2893959 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:05.491226 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac352d5-a3d5-4145-a811-8ffe7f5d4829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:05.493018 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac352d5-a3d5-4145-a811-8ffe7f5d4829 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:05.493636 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac352d5-a3d5-4145-a811-8ffe7f5d4829 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:05.494793 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac352d5-a3d5-4145-a811-8ffe7f5d4829 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:05.510038 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac352d5-a3d5-4145-a811-8ffe7f5d4829 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:05.512184 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac352d5-a3d5-4145-a811-8ffe7f5d4829 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:05.536463 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-0ac352d5-a3d5-4145-a811-8ffe7f5d4829 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:05.536463 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:05.536463 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:05.536463 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:05.536463 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:05.536463 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:05.536463 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:05.536463 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:05.536463 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:05.536463 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:05.536463 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:05.536463 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:05.536463 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:05.536463 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:05.536463 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:05.537501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:05.537501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:05.537501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:05.537501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:05.537501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:05.537501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:05.537501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:05.537501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:05.537501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:05.537501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:05.537501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:05.537501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:05.537501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:05.537501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:05.537501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:05.537501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:05.537501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:05.538385 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:05.538385 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:05.538385 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:05.538385 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:05.539913 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 343/672] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:05.542225 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac352d5-a3d5-4145-a811-8ffe7f5d4829 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:05.563280 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 330/673] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:21:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:21:05.564513 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0e34c-471f-4040-94ed-5e3f57c7e2de None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:05.627564 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:05.655060 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac352d5-a3d5-4145-a811-8ffe7f5d4829 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:05.666779 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:05.675927 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0e34c-471f-4040-94ed-5e3f57c7e2de None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:05.693567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1077504-1dda-4ce3-9d21-ddb39d8458a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:05.694633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1077504-1dda-4ce3-9d21-ddb39d8458a4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:05.694942 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1077504-1dda-4ce3-9d21-ddb39d8458a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:05.695227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1077504-1dda-4ce3-9d21-ddb39d8458a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:05.702659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1077504-1dda-4ce3-9d21-ddb39d8458a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:05.704427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1077504-1dda-4ce3-9d21-ddb39d8458a4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:05.711613 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42f8c0-42d7-4586-9dfe-f565c775c033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:05.712289 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42f8c0-42d7-4586-9dfe-f565c775c033 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:05.712580 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42f8c0-42d7-4586-9dfe-f565c775c033 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:05.713387 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f42f8c0-42d7-4586-9dfe-f565c775c033 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:05.714125 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42f8c0-42d7-4586-9dfe-f565c775c033 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:05.714446 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42f8c0-42d7-4586-9dfe-f565c775c033 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:05.715854 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f42f8c0-42d7-4586-9dfe-f565c775c033 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:05.720196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f1077504-1dda-4ce3-9d21-ddb39d8458a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:05.720196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:05.720196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:05.720196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:05.720196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:05.720196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:05.720196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:05.720196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:05.720196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:05.720196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:05.720196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:05.720196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:05.720196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:21:05.720196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:21:05.720196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:21:05.721070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:21:05.721070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:05.721070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:05.721070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:05.721070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:05.721070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:05.721070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:05.721070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:05.721070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:05.721070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:05.721070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:05.721070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:05.721070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:05.721070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:05.721070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:05.721070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:05.721070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:05.721968 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:21:05.721968 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:21:05.721968 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:21:05.721968 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:21:05.721968 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:21:05.721968 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:21:05.721968 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:05.721968 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:05.721968 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 344/674] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:21:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:05.722909 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1077504-1dda-4ce3-9d21-ddb39d8458a4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:05.735680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 331/675] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:05.736547 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42f8c0-42d7-4586-9dfe-f565c775c033 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:05.851052 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1077504-1dda-4ce3-9d21-ddb39d8458a4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:05.868834 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f42f8c0-42d7-4586-9dfe-f565c775c033 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:05.885516 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:05.889295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f6c76-70c2-4458-af83-d833bb3d4381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:05.890376 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f6c76-70c2-4458-af83-d833bb3d4381 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:05.890923 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f6c76-70c2-4458-af83-d833bb3d4381 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:05.891423 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f6c76-70c2-4458-af83-d833bb3d4381 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:05.903182 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f6c76-70c2-4458-af83-d833bb3d4381 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:05.904961 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f6c76-70c2-4458-af83-d833bb3d4381 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:05.908640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c19c25-3638-40d7-b9ca-972b1c2373da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:05.909312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c19c25-3638-40d7-b9ca-972b1c2373da None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:05.909604 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c19c25-3638-40d7-b9ca-972b1c2373da None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:05.909885 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c19c25-3638-40d7-b9ca-972b1c2373da None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:05.910709 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c19c25-3638-40d7-b9ca-972b1c2373da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:05.911026 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c19c25-3638-40d7-b9ca-972b1c2373da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:05.912449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c19c25-3638-40d7-b9ca-972b1c2373da None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:05.925827 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 332/676] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:21:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:05.926267 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ca3f6c76-70c2-4458-af83-d833bb3d4381 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:05.926267 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:05.926267 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:05.926267 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:05.926267 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:05.926267 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:05.926267 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:05.926267 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:05.926267 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:05.926267 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:05.926267 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:05.926267 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:05.926267 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:05.926267 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:05.926267 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:05.927221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:05.927221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:05.927221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:05.927221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:05.927221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:05.927221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:05.927221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:05.927221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:05.927221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:05.927221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:05.927221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:05.927221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:05.927221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:05.927221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:05.927221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:05.927221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:05.927221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:05.928117 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:05.928117 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:05.928117 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:05.928117 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:05.928338 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c19c25-3638-40d7-b9ca-972b1c2373da None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:05.930401 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 345/677] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:05.931410 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f6c76-70c2-4458-af83-d833bb3d4381 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:05.954660 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:06.034911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c19c25-3638-40d7-b9ca-972b1c2373da None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:06.039786 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f6c76-70c2-4458-af83-d833bb3d4381 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:06.071055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d7c76-ba5d-4cb4-82e1-977188be385a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:06.072580 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d7c76-ba5d-4cb4-82e1-977188be385a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:06.072761 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d7c76-ba5d-4cb4-82e1-977188be385a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:06.073007 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230d7c76-ba5d-4cb4-82e1-977188be385a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:06.077892 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c3ff5-41fb-4dd1-bdd3-03cdee17064c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:06.079197 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c3ff5-41fb-4dd1-bdd3-03cdee17064c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:06.079688 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c3ff5-41fb-4dd1-bdd3-03cdee17064c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:06.080169 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313c3ff5-41fb-4dd1-bdd3-03cdee17064c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:06.080939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d7c76-ba5d-4cb4-82e1-977188be385a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:06.081440 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c3ff5-41fb-4dd1-bdd3-03cdee17064c None admin] RBAC: Adding query filter params (role.id=8883c151055d4ba493f514ce1bb62e78, scope.project.id=f043ade9e4f748d4adc010ffb0a47a46, user.id=356d396ae5cf47c3bc61f24f9e6dfa02) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:06.081970 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c3ff5-41fb-4dd1-bdd3-03cdee17064c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:06.082479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230d7c76-ba5d-4cb4-82e1-977188be385a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:06.083850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313c3ff5-41fb-4dd1-bdd3-03cdee17064c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:06.091407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 346/678] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:21:05 2022] GET /identity/v3/role_assignments?user.id=356d396ae5cf47c3bc61f24f9e6dfa02&scope.project.id=f043ade9e4f748d4adc010ffb0a47a46&role.id=8883c151055d4ba493f514ce1bb62e78 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:06.092460 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c3ff5-41fb-4dd1-bdd3-03cdee17064c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:06.098584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-230d7c76-ba5d-4cb4-82e1-977188be385a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 12:21:06.098584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:06.098584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:06.098584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:06.098584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:06.098584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:06.098584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:06.098584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:06.098584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:06.098584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:06.098584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:06.098584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:06.098584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:21:06.098584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:21:06.098584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:21:06.098584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:21:06.099610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:06.099610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:06.099610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:21:06.099610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:06.099610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:21:06.099610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:06.099610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:06.099610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:06.099610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:06.099610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:06.099610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:06.099610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:06.099610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:06.099610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:06.099610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:06.099610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:06.099610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:06.099610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:06.100567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:06.100567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:06.100567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:21:06.100567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:21:06.100567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:21:06.100567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:21:06.100567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:21:06.100567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:21:06.100567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 12:21:06.100567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:06.100567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 333/679] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:21:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:06.115624 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d7c76-ba5d-4cb4-82e1-977188be385a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:06.155303 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:06.195077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:06.215052 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313c3ff5-41fb-4dd1-bdd3-03cdee17064c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:06.228020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230d7c76-ba5d-4cb4-82e1-977188be385a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:06.258391 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac196a-17da-439a-89a9-f377c85417bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:06.259099 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac196a-17da-439a-89a9-f377c85417bf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:06.259385 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac196a-17da-439a-89a9-f377c85417bf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:06.260236 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac196a-17da-439a-89a9-f377c85417bf None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:06.260984 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac196a-17da-439a-89a9-f377c85417bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:06.261290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac196a-17da-439a-89a9-f377c85417bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:06.262692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac196a-17da-439a-89a9-f377c85417bf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:06.276691 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 347/680] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:06.283650 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac196a-17da-439a-89a9-f377c85417bf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:06.290668 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d43658-8d0a-40cb-b317-502bc2968721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:06.291392 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d43658-8d0a-40cb-b317-502bc2968721 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:06.291685 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d43658-8d0a-40cb-b317-502bc2968721 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:06.291970 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d43658-8d0a-40cb-b317-502bc2968721 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:06.292838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d43658-8d0a-40cb-b317-502bc2968721 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:06.293136 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d43658-8d0a-40cb-b317-502bc2968721 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:06.294629 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d43658-8d0a-40cb-b317-502bc2968721 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:06.318259 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 334/681] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Jun 29 12:21:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:06.319451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccb8ca-5d53-4f37-a95b-ad74bb9abc42 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:06.319735 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccb8ca-5d53-4f37-a95b-ad74bb9abc42 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:06.320110 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccb8ca-5d53-4f37-a95b-ad74bb9abc42 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:06.321476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 335/682] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:06.326375 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cccb8ca-5d53-4f37-a95b-ad74bb9abc42 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:06.410640 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac196a-17da-439a-89a9-f377c85417bf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:06.445301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f206ce-d8e9-481a-89db-19e206f305d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:06.446220 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f206ce-d8e9-481a-89db-19e206f305d7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:06.446635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f206ce-d8e9-481a-89db-19e206f305d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:06.446989 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f206ce-d8e9-481a-89db-19e206f305d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:06.448009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:06.456357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f206ce-d8e9-481a-89db-19e206f305d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:06.458004 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f206ce-d8e9-481a-89db-19e206f305d7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:06.459498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cccb8ca-5d53-4f37-a95b-ad74bb9abc42 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:06.474385 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-11f206ce-d8e9-481a-89db-19e206f305d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:06.474385 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:06.474385 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:06.474385 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:06.474385 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:06.474385 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:06.474385 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:06.474385 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:06.474385 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:06.474385 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:06.474385 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:06.474385 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:06.474385 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:21:06.474385 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:21:06.474385 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:21:06.475522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:21:06.475522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:06.475522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:06.475522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:06.475522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:06.475522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:06.475522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:06.475522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:06.475522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:06.475522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:06.475522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:06.475522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:06.475522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:06.475522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:06.475522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:06.475522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:06.475522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:06.476628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:21:06.476628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:21:06.476628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:21:06.476628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:21:06.476628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:21:06.476628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:21:06.476628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:06.476628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:06.482556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 348/683] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:21:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:06.484201 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b8cc6-7fd1-4dd0-8960-1fe6f4854d0f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:06.484504 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b8cc6-7fd1-4dd0-8960-1fe6f4854d0f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:06.484779 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143b8cc6-7fd1-4dd0-8960-1fe6f4854d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:06.499220 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:06.504076 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ab416-e3d3-4f36-a383-6b4dac82fa1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:06.504219 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-143b8cc6-7fd1-4dd0-8960-1fe6f4854d0f None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:06.505095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ab416-e3d3-4f36-a383-6b4dac82fa1a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:06.505479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-143b8cc6-7fd1-4dd0-8960-1fe6f4854d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:06.505817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ab416-e3d3-4f36-a383-6b4dac82fa1a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:06.506276 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8ab416-e3d3-4f36-a383-6b4dac82fa1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:06.596287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 349/684] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:21:06.596480 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b8cc6-7fd1-4dd0-8960-1fe6f4854d0f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:06.610303 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ab416-e3d3-4f36-a383-6b4dac82fa1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:06.611804 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8ab416-e3d3-4f36-a383-6b4dac82fa1a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:06.713795 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143b8cc6-7fd1-4dd0-8960-1fe6f4854d0f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:06.714389 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:06.726327 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 336/685] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:21:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:21:06.727192 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ab416-e3d3-4f36-a383-6b4dac82fa1a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:06.754082 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4dc51-4512-4ba5-992c-ef03fdc49324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:06.755157 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4dc51-4512-4ba5-992c-ef03fdc49324 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:06.755650 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4dc51-4512-4ba5-992c-ef03fdc49324 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:06.756124 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4dc51-4512-4ba5-992c-ef03fdc49324 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:06.757269 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4dc51-4512-4ba5-992c-ef03fdc49324 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:06.757774 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4dc51-4512-4ba5-992c-ef03fdc49324 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:06.759553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4dc51-4512-4ba5-992c-ef03fdc49324 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:06.761823 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:06.780096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 350/686] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:21:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:06.781287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4dc51-4512-4ba5-992c-ef03fdc49324 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:06.841301 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8ab416-e3d3-4f36-a383-6b4dac82fa1a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:06.887645 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc354a3-49d2-4497-a164-2a7f7fc972fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:06.888448 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc354a3-49d2-4497-a164-2a7f7fc972fb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:06.888743 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc354a3-49d2-4497-a164-2a7f7fc972fb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:06.888846 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4dc51-4512-4ba5-992c-ef03fdc49324 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:06.889023 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc354a3-49d2-4497-a164-2a7f7fc972fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:06.900894 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc354a3-49d2-4497-a164-2a7f7fc972fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:06.902369 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc354a3-49d2-4497-a164-2a7f7fc972fb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:06.922741 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2cc354a3-49d2-4497-a164-2a7f7fc972fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:06.922741 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:06.922741 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:06.922741 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:06.922741 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:06.922741 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:06.922741 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:06.922741 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:06.922741 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:06.922741 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:06.922741 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:06.922741 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:06.922741 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:06.922741 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:06.922741 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:06.923705 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:06.923705 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:06.923705 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:06.923705 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:06.923705 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:06.923705 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:06.923705 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:06.923705 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:06.923705 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:06.923705 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:06.923705 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:06.923705 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:06.923705 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:06.923705 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:06.923705 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:06.923705 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:06.923705 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:06.924603 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:06.924603 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:06.924603 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:06.924603 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:06.928135 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97babde1-2073-49c1-b16e-d980694fce15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:06.928965 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 337/687] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:06.929469 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97babde1-2073-49c1-b16e-d980694fce15 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:06.929856 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc354a3-49d2-4497-a164-2a7f7fc972fb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:06.930168 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97babde1-2073-49c1-b16e-d980694fce15 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:06.930690 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97babde1-2073-49c1-b16e-d980694fce15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:06.943277 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97babde1-2073-49c1-b16e-d980694fce15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:06.945019 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97babde1-2073-49c1-b16e-d980694fce15 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:06.963757 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-97babde1-2073-49c1-b16e-d980694fce15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:06.963757 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:06.963757 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:06.963757 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:06.963757 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:06.963757 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:06.963757 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:06.963757 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:06.963757 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:06.963757 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:06.963757 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:06.963757 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:06.963757 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:06.963757 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:06.963757 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:06.964761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:06.964761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:06.964761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:06.964761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:06.964761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:06.964761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:06.964761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:06.964761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:06.964761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:06.964761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:06.964761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:06.964761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:06.964761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:06.964761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:06.964761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:06.964761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:06.964761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:06.965666 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:06.965666 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:06.965666 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:06.965666 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:06.968000 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 351/688] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:06.969048 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97babde1-2073-49c1-b16e-d980694fce15 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:06.979766 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:07.023257 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:07.043578 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc354a3-49d2-4497-a164-2a7f7fc972fb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:07.082742 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97babde1-2073-49c1-b16e-d980694fce15 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:07.085452 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8dcc5e-b14d-43b7-bbd8-601e84b047a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:07.086382 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dcc5e-b14d-43b7-bbd8-601e84b047a9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:07.086905 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dcc5e-b14d-43b7-bbd8-601e84b047a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:07.087389 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dcc5e-b14d-43b7-bbd8-601e84b047a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:07.089176 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8dcc5e-b14d-43b7-bbd8-601e84b047a9 None admin] RBAC: Adding query filter params (role.id=8883c151055d4ba493f514ce1bb62e78, scope.project.id=f043ade9e4f748d4adc010ffb0a47a46, user.id=ce922cdbf2d34f10b6e49631f0c6d770) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:07.089706 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8dcc5e-b14d-43b7-bbd8-601e84b047a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:07.091595 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8dcc5e-b14d-43b7-bbd8-601e84b047a9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:07.098749 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 338/689] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:21:06 2022] GET /identity/v3/role_assignments?user.id=ce922cdbf2d34f10b6e49631f0c6d770&scope.project.id=f043ade9e4f748d4adc010ffb0a47a46&role.id=8883c151055d4ba493f514ce1bb62e78 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:07.099809 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8dcc5e-b14d-43b7-bbd8-601e84b047a9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:07.138645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead5788-06b8-4fef-b2a2-2b4c2eb05a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:07.139357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead5788-06b8-4fef-b2a2-2b4c2eb05a51 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:07.139662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead5788-06b8-4fef-b2a2-2b4c2eb05a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:07.140016 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead5788-06b8-4fef-b2a2-2b4c2eb05a51 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:07.140746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aead5788-06b8-4fef-b2a2-2b4c2eb05a51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:07.141056 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aead5788-06b8-4fef-b2a2-2b4c2eb05a51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:07.142566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aead5788-06b8-4fef-b2a2-2b4c2eb05a51 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:07.164135 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 352/690] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:07.172118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead5788-06b8-4fef-b2a2-2b4c2eb05a51 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:07.222949 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8dcc5e-b14d-43b7-bbd8-601e84b047a9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:07.242829 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:07.283509 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07d7b4-2504-40d9-9e03-20ed2679809f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:07.284330 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07d7b4-2504-40d9-9e03-20ed2679809f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:07.284807 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07d7b4-2504-40d9-9e03-20ed2679809f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:07.285312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d07d7b4-2504-40d9-9e03-20ed2679809f None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:07.285972 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07d7b4-2504-40d9-9e03-20ed2679809f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:07.286286 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07d7b4-2504-40d9-9e03-20ed2679809f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:07.287747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d07d7b4-2504-40d9-9e03-20ed2679809f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:07.301648 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 339/691] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:07.307513 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07d7b4-2504-40d9-9e03-20ed2679809f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:07.382670 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aead5788-06b8-4fef-b2a2-2b4c2eb05a51 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:07.391990 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:07.428096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8cfac-33bf-40a3-8d42-2cd7d61dc9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:07.428778 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8cfac-33bf-40a3-8d42-2cd7d61dc9be None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:07.429068 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8cfac-33bf-40a3-8d42-2cd7d61dc9be None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:07.429343 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8cfac-33bf-40a3-8d42-2cd7d61dc9be None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:07.430056 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae8cfac-33bf-40a3-8d42-2cd7d61dc9be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:07.432504 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae8cfac-33bf-40a3-8d42-2cd7d61dc9be None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:07.441683 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d07d7b4-2504-40d9-9e03-20ed2679809f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:07.457844 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 353/692] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 12:21:07 2022] POST /identity/v3/users => generated 254 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:21:07.485640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4fb58c-d791-4cb5-8077-52bce90ca53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:07.486447 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fb58c-d791-4cb5-8077-52bce90ca53b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:07.486766 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fb58c-d791-4cb5-8077-52bce90ca53b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:07.487039 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fb58c-d791-4cb5-8077-52bce90ca53b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:07.496613 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4fb58c-d791-4cb5-8077-52bce90ca53b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:07.498092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4fb58c-d791-4cb5-8077-52bce90ca53b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:07.513600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8d4fb58c-d791-4cb5-8077-52bce90ca53b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:07.513600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:07.513600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:07.513600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:07.513600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:07.513600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:07.513600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:07.513600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:07.513600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:07.513600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:07.513600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:07.513600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:07.513600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:21:07.513600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:21:07.513600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:21:07.514589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:21:07.514589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:07.514589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:07.514589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:07.514589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:07.514589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:07.514589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:07.514589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:07.514589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:07.514589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:07.514589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:07.514589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:07.514589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:07.514589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:07.514589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:07.514589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:07.514589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:07.515503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:21:07.515503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:21:07.515503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:21:07.515503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:21:07.515503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:21:07.515503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:21:07.515503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:07.515503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:07.515503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 340/693] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:21:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:07.520735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8cfac-33bf-40a3-8d42-2cd7d61dc9be None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:07.597102 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88eaf26-fb74-47c5-b4cc-8e88a1aeb6db None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:07.597750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88eaf26-fb74-47c5-b4cc-8e88a1aeb6db None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:07.598337 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88eaf26-fb74-47c5-b4cc-8e88a1aeb6db None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:07.605186 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 341/694] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:07.611900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021331ca-4e3e-4b95-9aec-3b87a35293c2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:07.612461 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021331ca-4e3e-4b95-9aec-3b87a35293c2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:07.612920 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021331ca-4e3e-4b95-9aec-3b87a35293c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:07.641490 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-021331ca-4e3e-4b95-9aec-3b87a35293c2 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:07.642885 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-021331ca-4e3e-4b95-9aec-3b87a35293c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:07.657713 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8cfac-33bf-40a3-8d42-2cd7d61dc9be None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:07.676930 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:07.699099 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b504ba-8284-4780-82f3-72e8562fd62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:07.699766 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b504ba-8284-4780-82f3-72e8562fd62a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:07.700048 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b504ba-8284-4780-82f3-72e8562fd62a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:07.700325 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b504ba-8284-4780-82f3-72e8562fd62a None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:07.701111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b504ba-8284-4780-82f3-72e8562fd62a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:07.701418 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b504ba-8284-4780-82f3-72e8562fd62a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:07.703013 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b504ba-8284-4780-82f3-72e8562fd62a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:07.718280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 354/695] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:21:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:07.724780 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b504ba-8284-4780-82f3-72e8562fd62a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:07.735038 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 342/696] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:21:07.755116 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:07.763342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021331ca-4e3e-4b95-9aec-3b87a35293c2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:07.846347 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b504ba-8284-4780-82f3-72e8562fd62a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:07.875677 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021331ca-4e3e-4b95-9aec-3b87a35293c2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:07.886331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8699252-39c2-45d4-a280-06a4a85a7b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:07.887367 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8699252-39c2-45d4-a280-06a4a85a7b12 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:07.887946 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8699252-39c2-45d4-a280-06a4a85a7b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:07.888453 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8699252-39c2-45d4-a280-06a4a85a7b12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:07.889535 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8699252-39c2-45d4-a280-06a4a85a7b12 None admin] RBAC: Adding query filter params (role.id=8883c151055d4ba493f514ce1bb62e78, scope.project.id=f043ade9e4f748d4adc010ffb0a47a46, user.id=209b0df45f76404da880a9ae0c637d67) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:07.890091 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8699252-39c2-45d4-a280-06a4a85a7b12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:07.891913 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8699252-39c2-45d4-a280-06a4a85a7b12 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:07.899460 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 355/697] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:21:07 2022] GET /identity/v3/role_assignments?user.id=209b0df45f76404da880a9ae0c637d67&scope.project.id=f043ade9e4f748d4adc010ffb0a47a46&role.id=8883c151055d4ba493f514ce1bb62e78 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:07.917698 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feff255c-ee03-4cf7-afbd-ebc9bf05d130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:07.918581 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff255c-ee03-4cf7-afbd-ebc9bf05d130 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:07.918876 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff255c-ee03-4cf7-afbd-ebc9bf05d130 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:07.919157 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff255c-ee03-4cf7-afbd-ebc9bf05d130 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:07.932622 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feff255c-ee03-4cf7-afbd-ebc9bf05d130 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:21:07.942275 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-feff255c-ee03-4cf7-afbd-ebc9bf05d130 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:21:07.942275 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:07.942275 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:07.942275 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:07.942275 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:07.942275 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:07.942275 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:07.942275 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:07.942275 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:07.942275 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:07.942275 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:07.942275 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:07.942275 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:21:07.942275 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:21:07.942275 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:21:07.942275 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:21:07.943273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:21:07.943273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:21:07.943273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:21:07.943273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:21:07.943273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:07.943273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:07.943273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:07.943273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:07.943273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:07.943273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:07.943273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:07.943273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:07.943273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:07.943273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:07.943273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:07.943273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:07.944397 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:07.944397 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:07.944397 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:07.944397 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:07.944397 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:21:07.944397 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:21:07.944397 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:21:07.944397 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:21:07.944397 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:21:07.944397 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:21:07.944397 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:21:07.944397 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:07.944397 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 343/698] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:21:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:07.951888 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8699252-39c2-45d4-a280-06a4a85a7b12 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:08.082836 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:08.099676 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8699252-39c2-45d4-a280-06a4a85a7b12 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:08.134634 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2cabdc-05e1-4afe-a019-e3572e8d4fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:08.135558 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cabdc-05e1-4afe-a019-e3572e8d4fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:08.135983 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cabdc-05e1-4afe-a019-e3572e8d4fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:08.136424 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cabdc-05e1-4afe-a019-e3572e8d4fc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:08.137322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2cabdc-05e1-4afe-a019-e3572e8d4fc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:08.137772 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2cabdc-05e1-4afe-a019-e3572e8d4fc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:08.139310 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2cabdc-05e1-4afe-a019-e3572e8d4fc9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:08.152962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 356/699] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:21:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:08.159553 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feff255c-ee03-4cf7-afbd-ebc9bf05d130 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:08.180951 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:08.268425 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feff255c-ee03-4cf7-afbd-ebc9bf05d130 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:08.309867 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93bfb8-519a-46a7-8c79-9a8e09f6a854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:08.310696 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93bfb8-519a-46a7-8c79-9a8e09f6a854 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:08.310983 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93bfb8-519a-46a7-8c79-9a8e09f6a854 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:08.311259 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93bfb8-519a-46a7-8c79-9a8e09f6a854 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:08.326267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f93bfb8-519a-46a7-8c79-9a8e09f6a854 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:08.327759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f93bfb8-519a-46a7-8c79-9a8e09f6a854 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:08.348900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7f93bfb8-519a-46a7-8c79-9a8e09f6a854 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:08.348900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:08.348900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:08.348900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:08.348900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:08.348900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:08.348900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:08.348900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:08.348900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:08.348900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:08.348900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:08.348900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:08.348900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:08.348900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:08.348900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:08.349824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:08.349824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:08.349824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:08.349824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:08.349824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:08.349824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:08.349824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:08.349824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:08.349824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:08.349824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:08.349824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:08.349824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:08.349824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:08.349824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:08.349824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:08.349824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:08.349824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:08.351021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:08.351021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:08.351021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:08.351021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:08.351021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 344/700] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:08.357165 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2cabdc-05e1-4afe-a019-e3572e8d4fc9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:08.438284 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff37b3-888c-4f3c-a172-f17a88c755ae None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:08.438633 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff37b3-888c-4f3c-a172-f17a88c755ae None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:08.438882 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff37b3-888c-4f3c-a172-f17a88c755ae None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:08.440372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 345/701] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:21:08.445742 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e8ea7-1cbb-4579-9c96-3f686c606d00 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:08.446241 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e8ea7-1cbb-4579-9c96-3f686c606d00 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:08.446757 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e8ea7-1cbb-4579-9c96-3f686c606d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:08.464497 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2cabdc-05e1-4afe-a019-e3572e8d4fc9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:08.467480 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-315e8ea7-1cbb-4579-9c96-3f686c606d00 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:08.468932 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-315e8ea7-1cbb-4579-9c96-3f686c606d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:08.500804 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db77133-37e4-45e8-b72c-5a1025cef9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:08.501486 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db77133-37e4-45e8-b72c-5a1025cef9f8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:08.501769 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db77133-37e4-45e8-b72c-5a1025cef9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:08.502047 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db77133-37e4-45e8-b72c-5a1025cef9f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:08.503262 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db77133-37e4-45e8-b72c-5a1025cef9f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:08.503262 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db77133-37e4-45e8-b72c-5a1025cef9f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:08.504453 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db77133-37e4-45e8-b72c-5a1025cef9f8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:08.522525 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 357/702] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:08.524455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db77133-37e4-45e8-b72c-5a1025cef9f8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:08.537847 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:08.552108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 346/703] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:21:08.554810 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:08.578191 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e8ea7-1cbb-4579-9c96-3f686c606d00 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:08.646860 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db77133-37e4-45e8-b72c-5a1025cef9f8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:08.683996 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b5426-bce0-4817-8f28-5293b88fa75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:08.684787 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b5426-bce0-4817-8f28-5293b88fa75d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:08.685070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b5426-bce0-4817-8f28-5293b88fa75d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:08.685346 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b5426-bce0-4817-8f28-5293b88fa75d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:08.693145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b5426-bce0-4817-8f28-5293b88fa75d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:08.694718 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b5426-bce0-4817-8f28-5293b88fa75d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:08.698707 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e8ea7-1cbb-4579-9c96-3f686c606d00 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:08.710924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-386b5426-bce0-4817-8f28-5293b88fa75d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 12:21:08.710924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:08.710924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:08.710924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:08.710924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:08.710924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:08.710924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:08.710924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:08.710924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:08.710924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:08.710924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:08.710924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:08.710924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:21:08.710924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:21:08.710924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:21:08.710924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:21:08.712020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:08.712020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:08.712020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:21:08.712020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:08.712020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:21:08.712020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:08.712020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:08.712020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:08.712020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:08.712020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:08.712020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:08.712020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:08.712020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:08.712020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:08.712020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:08.712020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:08.712020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:08.712020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:08.713519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:08.713519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:08.713519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:21:08.713519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:21:08.713519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:21:08.713519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:21:08.713519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:21:08.713519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:21:08.713519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 12:21:08.713519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:08.713519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 358/704] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Jun 29 12:21:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:08.718372 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b5426-bce0-4817-8f28-5293b88fa75d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:08.738673 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd0e83-5bab-493e-acda-7bfe640c4879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:08.739650 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd0e83-5bab-493e-acda-7bfe640c4879 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:08.740228 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd0e83-5bab-493e-acda-7bfe640c4879 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:08.740511 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd0e83-5bab-493e-acda-7bfe640c4879 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:08.741172 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd0e83-5bab-493e-acda-7bfe640c4879 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:08.742566 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd0e83-5bab-493e-acda-7bfe640c4879 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:08.762664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1fbd0e83-5bab-493e-acda-7bfe640c4879 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 12:21:08.762664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:08.762664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 12:21:08.762664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 12:21:08.762664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 12:21:08.762664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 12:21:08.762664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 12:21:08.762664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:08.762664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 12:21:08.762664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:08.762664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:21:08.762664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:08.762664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:08.762664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:08.762664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:08.762664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:08.764996 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:08.764996 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:08.764996 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:08.764996 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:08.764996 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:08.764996 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:08.764996 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:08.764996 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 12:21:08.764996 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 12:21:08.764996 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 12:21:08.764996 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 12:21:08.764996 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:08.764996 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:08.764996 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:08.764996 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:08.764996 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:08.766968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:08.766968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:08.766968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:08.766968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:08.766968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:08.766968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:08.766968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:08.766968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:08.766968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:08.766968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:08.766968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:08.766968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 12:21:08.766968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:08.766968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 12:21:08.766968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:08.766968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 347/705] 10.222.0.52 () {56 vars in 1129 bytes} [Wed Jun 29 12:21:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:08.773607 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd0e83-5bab-493e-acda-7bfe640c4879 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:08.801116 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:08.842063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b5426-bce0-4817-8f28-5293b88fa75d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:08.861515 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:08.909499 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd0e83-5bab-493e-acda-7bfe640c4879 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:08.923109 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a6891-b962-4f7b-b9af-8b04f599a886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:08.923791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a6891-b962-4f7b-b9af-8b04f599a886 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:08.924078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a6891-b962-4f7b-b9af-8b04f599a886 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:08.924357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a6891-b962-4f7b-b9af-8b04f599a886 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:08.925196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a6891-b962-4f7b-b9af-8b04f599a886 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:08.925508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a6891-b962-4f7b-b9af-8b04f599a886 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:08.927037 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a6891-b962-4f7b-b9af-8b04f599a886 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:08.952442 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1bdaed-7d48-4d7a-a018-958d2faf9402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:08.953870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1bdaed-7d48-4d7a-a018-958d2faf9402 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:08.954383 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1bdaed-7d48-4d7a-a018-958d2faf9402 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:08.954964 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1bdaed-7d48-4d7a-a018-958d2faf9402 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:08.955877 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1bdaed-7d48-4d7a-a018-958d2faf9402 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:08.956391 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1bdaed-7d48-4d7a-a018-958d2faf9402 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:08.958055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1bdaed-7d48-4d7a-a018-958d2faf9402 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:08.961165 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 359/706] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Jun 29 12:21:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:08.967611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a6891-b962-4f7b-b9af-8b04f599a886 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:08.969203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 348/707] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Jun 29 12:21:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:08.975225 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1bdaed-7d48-4d7a-a018-958d2faf9402 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:09.082838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1bdaed-7d48-4d7a-a018-958d2faf9402 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:09.112344 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:09.128659 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1afe4-3ef9-4395-8cca-6db7acd38d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:09.129697 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1afe4-3ef9-4395-8cca-6db7acd38d94 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:09.130210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1afe4-3ef9-4395-8cca-6db7acd38d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:09.130738 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1afe4-3ef9-4395-8cca-6db7acd38d94 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:09.131660 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b1afe4-3ef9-4395-8cca-6db7acd38d94 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:09.132190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b1afe4-3ef9-4395-8cca-6db7acd38d94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:09.134176 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b1afe4-3ef9-4395-8cca-6db7acd38d94 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:09.149102 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 349/708] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Jun 29 12:21:08 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:09.171441 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a6891-b962-4f7b-b9af-8b04f599a886 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:09.217543 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:09.231761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3305579f-b386-4e1f-8eec-b703ea023da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:09.232435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305579f-b386-4e1f-8eec-b703ea023da8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:09.232719 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305579f-b386-4e1f-8eec-b703ea023da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:09.233005 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305579f-b386-4e1f-8eec-b703ea023da8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:09.322312 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3305579f-b386-4e1f-8eec-b703ea023da8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:09.323792 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3305579f-b386-4e1f-8eec-b703ea023da8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:09.471850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 360/709] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:21:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 29 12:21:09.477592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1afe4-3ef9-4395-8cca-6db7acd38d94 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:09.591110 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1afe4-3ef9-4395-8cca-6db7acd38d94 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:09.627436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acdb8a9-ff39-457a-a2d5-99bbc3543dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:09.628637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acdb8a9-ff39-457a-a2d5-99bbc3543dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:09.629132 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acdb8a9-ff39-457a-a2d5-99bbc3543dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:09.629614 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acdb8a9-ff39-457a-a2d5-99bbc3543dc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:09.647788 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acdb8a9-ff39-457a-a2d5-99bbc3543dc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:09.649598 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acdb8a9-ff39-457a-a2d5-99bbc3543dc6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:09.670938 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3acdb8a9-ff39-457a-a2d5-99bbc3543dc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:09.670938 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:09.670938 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:09.670938 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:09.670938 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:09.670938 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:09.670938 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:09.670938 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:09.670938 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:09.670938 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:09.670938 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:09.670938 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:09.670938 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:09.670938 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:09.670938 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:09.671903 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:09.671903 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:09.671903 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:09.671903 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:09.671903 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:09.671903 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:09.671903 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:09.671903 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:09.671903 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:09.671903 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:09.671903 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:09.671903 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:09.671903 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:09.671903 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:09.671903 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:09.671903 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:09.671903 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:09.672888 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:09.672888 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:09.672888 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:09.672888 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:09.675312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 350/710] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:09.683556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3305579f-b386-4e1f-8eec-b703ea023da8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:09.735237 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574f7d4-52aa-4cb6-9344-f159cbea2109 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:09.735534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574f7d4-52aa-4cb6-9344-f159cbea2109 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:09.735720 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574f7d4-52aa-4cb6-9344-f159cbea2109 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:09.736266 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:09.738292 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 351/711] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:21:09.742544 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa2abc-0b7e-45bc-9f7e-518f9572a2ba None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:09.742837 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa2abc-0b7e-45bc-9f7e-518f9572a2ba None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:09.743115 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa2abc-0b7e-45bc-9f7e-518f9572a2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:09.770550 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3bfa2abc-0b7e-45bc-9f7e-518f9572a2ba None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:09.772014 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3bfa2abc-0b7e-45bc-9f7e-518f9572a2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:09.819111 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:09.858040 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3305579f-b386-4e1f-8eec-b703ea023da8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:09.870054 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 352/712] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:21:09.907882 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e809b-e4c7-4e7a-a133-c630719eb75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:09.908549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e809b-e4c7-4e7a-a133-c630719eb75f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:09.908831 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e809b-e4c7-4e7a-a133-c630719eb75f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:09.909103 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e809b-e4c7-4e7a-a133-c630719eb75f None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:09.909823 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e809b-e4c7-4e7a-a133-c630719eb75f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:09.910126 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e809b-e4c7-4e7a-a133-c630719eb75f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:09.911521 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e809b-e4c7-4e7a-a133-c630719eb75f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:09.924380 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa2abc-0b7e-45bc-9f7e-518f9572a2ba None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:09.927910 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 361/713] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:09.934332 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e809b-e4c7-4e7a-a133-c630719eb75f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:10.077667 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:10.100361 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e809b-e4c7-4e7a-a133-c630719eb75f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:10.113884 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa2abc-0b7e-45bc-9f7e-518f9572a2ba None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:10.169511 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2162bd1-8309-41d9-a5b8-ac034b49a4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:10.170314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2162bd1-8309-41d9-a5b8-ac034b49a4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:10.170631 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2162bd1-8309-41d9-a5b8-ac034b49a4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:10.170912 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2162bd1-8309-41d9-a5b8-ac034b49a4d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:10.182225 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c08e37-c327-4c1b-8ed6-a1f41e1ac4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:10.183165 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c08e37-c327-4c1b-8ed6-a1f41e1ac4c4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:10.183454 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c08e37-c327-4c1b-8ed6-a1f41e1ac4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:10.183732 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c08e37-c327-4c1b-8ed6-a1f41e1ac4c4 None admin] PATH_INFO: `/v3/services/network` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:10.184459 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c08e37-c327-4c1b-8ed6-a1f41e1ac4c4 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:10.185851 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c08e37-c327-4c1b-8ed6-a1f41e1ac4c4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:10.187345 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2162bd1-8309-41d9-a5b8-ac034b49a4d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:10.189943 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2162bd1-8309-41d9-a5b8-ac034b49a4d0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:10.207122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-16c08e37-c327-4c1b-8ed6-a1f41e1ac4c4 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 12:21:10.207122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:10.207122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 12:21:10.207122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 12:21:10.207122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 12:21:10.207122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 12:21:10.207122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 12:21:10.207122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:10.207122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 12:21:10.207122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:10.207122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:21:10.207122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:10.207122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:10.207122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:10.207122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:10.207122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:10.207122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:10.208182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:10.208182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:10.208182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:10.208182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:10.208182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:10.208182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:10.208182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 12:21:10.208182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 12:21:10.208182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 12:21:10.208182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 12:21:10.208182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:10.208182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:10.208182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:10.208182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:10.208182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:10.208182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:10.209037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:10.209037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:10.209037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:10.209037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:10.209037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:10.209037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:10.209037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:10.209037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:10.209037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:10.209037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:10.209037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 12:21:10.209037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:10.209037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 12:21:10.209037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:10.217641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a2162bd1-8309-41d9-a5b8-ac034b49a4d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:10.217641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:10.217641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:10.217641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:10.217641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:10.217641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:10.217641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:10.217641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:10.217641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:10.217641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:10.217641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:10.217641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:10.217641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:21:10.217641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:21:10.217641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:21:10.219740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:21:10.219740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:10.219740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:10.219740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:10.219740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:10.219740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:10.219740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:10.219740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:10.219740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:10.219740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:10.219740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:10.219740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:10.219740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:10.219740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:10.219740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:10.219740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:10.219740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:10.221947 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:21:10.221947 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:21:10.221947 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:21:10.221947 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:21:10.221947 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:21:10.221947 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:21:10.221947 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:10.221947 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:10.222967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 353/714] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:21:09 2022] GET /identity/v3/services/network => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:10.222967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c08e37-c327-4c1b-8ed6-a1f41e1ac4c4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:10.231696 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 362/715] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:21:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:10.236169 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2162bd1-8309-41d9-a5b8-ac034b49a4d0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:10.293971 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:10.359793 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c08e37-c327-4c1b-8ed6-a1f41e1ac4c4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:10.382514 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2162bd1-8309-41d9-a5b8-ac034b49a4d0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:10.397111 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9bc132-96a0-4fe5-a8e4-abf06100e9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:10.397782 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bc132-96a0-4fe5-a8e4-abf06100e9e0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:10.398073 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bc132-96a0-4fe5-a8e4-abf06100e9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:10.398357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9bc132-96a0-4fe5-a8e4-abf06100e9e0 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:10.399104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9bc132-96a0-4fe5-a8e4-abf06100e9e0 None admin] RBAC: Adding query filter params (name=network) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:10.399417 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9bc132-96a0-4fe5-a8e4-abf06100e9e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:10.400857 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9bc132-96a0-4fe5-a8e4-abf06100e9e0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:10.405650 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 354/716] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jun 29 12:21:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:10.406923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8e2d0-8bd3-4829-b8f9-0a235a9f7cc4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:10.407505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8e2d0-8bd3-4829-b8f9-0a235a9f7cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:10.408004 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8e2d0-8bd3-4829-b8f9-0a235a9f7cc4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:10.410446 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 355/717] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:10.412156 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8e2d0-8bd3-4829-b8f9-0a235a9f7cc4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:10.434015 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33172b54-7b6e-46a5-a48a-29aba1e444d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:10.434712 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33172b54-7b6e-46a5-a48a-29aba1e444d3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:10.435001 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33172b54-7b6e-46a5-a48a-29aba1e444d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:10.435281 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33172b54-7b6e-46a5-a48a-29aba1e444d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:10.436596 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33172b54-7b6e-46a5-a48a-29aba1e444d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:10.436916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33172b54-7b6e-46a5-a48a-29aba1e444d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:10.438919 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33172b54-7b6e-46a5-a48a-29aba1e444d3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:10.455497 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 363/718] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:21:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:10.457290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214fd8c-a0b5-4af8-ad10-6f330086a428 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:10.457574 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214fd8c-a0b5-4af8-ad10-6f330086a428 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:10.457857 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214fd8c-a0b5-4af8-ad10-6f330086a428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:10.471753 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:10.480810 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3214fd8c-a0b5-4af8-ad10-6f330086a428 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:10.482756 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3214fd8c-a0b5-4af8-ad10-6f330086a428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:10.529264 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8e2d0-8bd3-4829-b8f9-0a235a9f7cc4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:10.531007 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:10.571631 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc9345-236b-40f8-8eb0-821aeafb17ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:10.572898 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc9345-236b-40f8-8eb0-821aeafb17ac None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:10.573187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc9345-236b-40f8-8eb0-821aeafb17ac None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:10.573470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc9345-236b-40f8-8eb0-821aeafb17ac None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:10.574184 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc9345-236b-40f8-8eb0-821aeafb17ac None admin] RBAC: Adding query filter params (type=network) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:10.574539 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc9345-236b-40f8-8eb0-821aeafb17ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:10.575959 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc9345-236b-40f8-8eb0-821aeafb17ac None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:10.577933 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 364/719] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:21:10.579006 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214fd8c-a0b5-4af8-ad10-6f330086a428 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:10.594446 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 356/720] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jun 29 12:21:10 2022] GET /identity/v3/services?type=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:10.606443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc9345-236b-40f8-8eb0-821aeafb17ac None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:10.758625 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214fd8c-a0b5-4af8-ad10-6f330086a428 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:10.822858 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc9345-236b-40f8-8eb0-821aeafb17ac None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:10.823230 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1bb25-b641-464d-bf3f-7450221077b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:10.824229 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1bb25-b641-464d-bf3f-7450221077b2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:10.824954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1bb25-b641-464d-bf3f-7450221077b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:10.826402 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1bb25-b641-464d-bf3f-7450221077b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:10.827592 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1bb25-b641-464d-bf3f-7450221077b2 None admin] RBAC: Adding query filter params (role.id=3a7f4bf556344407a6d9540c89d13641, scope.project.id=f043ade9e4f748d4adc010ffb0a47a46, user.id=3ac7391a3c41499d94e5f97478797313) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:10.828126 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1bb25-b641-464d-bf3f-7450221077b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:10.829774 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded1bb25-b641-464d-bf3f-7450221077b2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:10.835278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:10.840718 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 365/721] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:21:10 2022] GET /identity/v3/role_assignments?user.id=3ac7391a3c41499d94e5f97478797313&scope.project.id=f043ade9e4f748d4adc010ffb0a47a46&role.id=3a7f4bf556344407a6d9540c89d13641 => generated 259 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:10.877561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e1edc-1a34-4d3d-a7d9-f1c70923215d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:10.878394 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e1edc-1a34-4d3d-a7d9-f1c70923215d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:10.878736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e1edc-1a34-4d3d-a7d9-f1c70923215d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:10.879010 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e1edc-1a34-4d3d-a7d9-f1c70923215d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:10.881971 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c62649-dad2-486b-92f7-688831ea2c1a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:10.882914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c62649-dad2-486b-92f7-688831ea2c1a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:10.883405 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c62649-dad2-486b-92f7-688831ea2c1a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:10.885493 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 366/722] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:21:10.890959 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40285ce9-8615-460c-9ff9-3c68d56603b0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:10.891562 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40285ce9-8615-460c-9ff9-3c68d56603b0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:10.892048 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40285ce9-8615-460c-9ff9-3c68d56603b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:10.896958 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e1edc-1a34-4d3d-a7d9-f1c70923215d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:21:10.906200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8d2e1edc-1a34-4d3d-a7d9-f1c70923215d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:21:10.906200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:10.906200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:10.906200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:10.906200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:10.906200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:10.906200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:10.906200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:10.906200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:10.906200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:10.906200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:10.906200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:10.906200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:21:10.906200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:21:10.906200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:21:10.906200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:21:10.907250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:21:10.907250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:21:10.907250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:21:10.907250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:21:10.907250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:10.907250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:10.907250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:10.907250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:10.907250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:10.907250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:10.907250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:10.907250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:10.907250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:10.907250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:10.907250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:10.907250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:10.908084 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:10.908084 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:10.908084 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:10.908084 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:10.908084 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:21:10.908084 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:21:10.908084 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:21:10.908084 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:21:10.908084 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:21:10.908084 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:21:10.908084 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:21:10.908084 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:10.908084 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 357/723] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 12:21:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:10.914190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e1edc-1a34-4d3d-a7d9-f1c70923215d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:10.915505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-40285ce9-8615-460c-9ff9-3c68d56603b0 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:10.916905 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-40285ce9-8615-460c-9ff9-3c68d56603b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:10.986910 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:11.035431 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 367/724] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 29 12:21:11.054460 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e1edc-1a34-4d3d-a7d9-f1c70923215d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:11.076464 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40285ce9-8615-460c-9ff9-3c68d56603b0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:11.115685 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077e512-ee2f-46eb-9118-c60acbdcc1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:11.116385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077e512-ee2f-46eb-9118-c60acbdcc1c7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:11.116645 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077e512-ee2f-46eb-9118-c60acbdcc1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:11.116924 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077e512-ee2f-46eb-9118-c60acbdcc1c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:11.117684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077e512-ee2f-46eb-9118-c60acbdcc1c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:11.117991 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077e512-ee2f-46eb-9118-c60acbdcc1c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:11.119952 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077e512-ee2f-46eb-9118-c60acbdcc1c7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:11.133864 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 358/725] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 12:21:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:11.139413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077e512-ee2f-46eb-9118-c60acbdcc1c7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:11.256137 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40285ce9-8615-460c-9ff9-3c68d56603b0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:11.292171 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:11.306167 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28db50-4a5f-47ff-a1f0-308e60cf55ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:11.308901 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28db50-4a5f-47ff-a1f0-308e60cf55ba None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:11.309192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28db50-4a5f-47ff-a1f0-308e60cf55ba None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:11.309476 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28db50-4a5f-47ff-a1f0-308e60cf55ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:11.327079 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:11.335010 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff28db50-4a5f-47ff-a1f0-308e60cf55ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:21:11.345734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ff28db50-4a5f-47ff-a1f0-308e60cf55ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:21:11.345734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:11.345734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:11.345734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:11.345734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:11.345734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:11.345734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:11.345734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:11.345734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:11.345734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:11.345734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:11.345734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:11.345734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:21:11.345734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:21:11.345734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:21:11.345734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:21:11.347311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:21:11.347311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:21:11.347311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:21:11.347311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:21:11.347311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:11.347311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:11.347311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:11.347311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:11.347311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:11.347311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:11.347311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:11.347311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:11.347311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:11.347311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:11.347311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:11.347311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:11.349284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:11.349284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:11.349284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:11.349284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:11.349284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:21:11.349284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:21:11.349284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:21:11.349284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:21:11.349284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:21:11.349284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:21:11.349284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:21:11.349284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:11.360168 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077e512-ee2f-46eb-9118-c60acbdcc1c7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:11.360340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 368/726] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:21:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:11.360340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28db50-4a5f-47ff-a1f0-308e60cf55ba None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:11.402170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e21aa3-6f70-43b8-b737-faae60a8e432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:11.403114 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e21aa3-6f70-43b8-b737-faae60a8e432 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:11.403406 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e21aa3-6f70-43b8-b737-faae60a8e432 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:11.403691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e21aa3-6f70-43b8-b737-faae60a8e432 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:11.433604 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e21aa3-6f70-43b8-b737-faae60a8e432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:11.435103 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e21aa3-6f70-43b8-b737-faae60a8e432 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:11.477976 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28db50-4a5f-47ff-a1f0-308e60cf55ba None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:11.483074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d1e21aa3-6f70-43b8-b737-faae60a8e432 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:11.483074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:11.483074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:11.483074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:11.483074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:11.483074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:11.483074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:11.483074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:11.483074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:11.483074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:11.483074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:11.483074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:11.483074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:11.483074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:11.483074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:11.483943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:11.483943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:11.483943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:11.483943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:11.483943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:11.483943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:11.483943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:11.483943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:11.483943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:11.483943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:11.483943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:11.483943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:11.483943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:11.483943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:11.483943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:11.483943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:11.483943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:11.484830 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:11.484830 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:11.484830 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:11.484830 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:11.484830 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 359/727] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:11.495753 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e21aa3-6f70-43b8-b737-faae60a8e432 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:11.515338 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2016b-ac31-4648-b8f6-4abf72864f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:11.515338 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2016b-ac31-4648-b8f6-4abf72864f66 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:11.515827 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2016b-ac31-4648-b8f6-4abf72864f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:11.515827 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2016b-ac31-4648-b8f6-4abf72864f66 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:11.516587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2016b-ac31-4648-b8f6-4abf72864f66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:11.516898 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2016b-ac31-4648-b8f6-4abf72864f66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:11.518320 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c2016b-ac31-4648-b8f6-4abf72864f66 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:11.528182 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 369/728] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:21:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:11.533318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2016b-ac31-4648-b8f6-4abf72864f66 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:11.592602 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:11.645773 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c2016b-ac31-4648-b8f6-4abf72864f66 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:11.679916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5188ceb-56fd-4e4e-b9e6-0b9b71d56ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:11.680708 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5188ceb-56fd-4e4e-b9e6-0b9b71d56ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:11.680989 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5188ceb-56fd-4e4e-b9e6-0b9b71d56ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:11.681282 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5188ceb-56fd-4e4e-b9e6-0b9b71d56ca7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:11.693211 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5188ceb-56fd-4e4e-b9e6-0b9b71d56ca7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:11.694708 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5188ceb-56fd-4e4e-b9e6-0b9b71d56ca7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:11.712584 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d5188ceb-56fd-4e4e-b9e6-0b9b71d56ca7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:11.712584 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:11.712584 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:11.712584 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:11.712584 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:11.712584 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:11.712584 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:11.712584 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:11.712584 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:11.712584 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:11.712584 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:11.712584 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:11.712584 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:11.712584 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:11.712584 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:11.714659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:11.714659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:11.714659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:11.714659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:11.714659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:11.714659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:11.714659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:11.714659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:11.714659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:11.714659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:11.714659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:11.714659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:11.714659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:11.714659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:11.714659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:11.714659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:11.714659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:11.716759 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:11.716759 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:11.716759 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:11.716759 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:11.718569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 370/729] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:11.719754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e21aa3-6f70-43b8-b737-faae60a8e432 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:11.719963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5188ceb-56fd-4e4e-b9e6-0b9b71d56ca7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:11.765792 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd791d-8b49-498c-900e-ae86c3be60d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:11.766490 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd791d-8b49-498c-900e-ae86c3be60d2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:11.766787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd791d-8b49-498c-900e-ae86c3be60d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:11.767666 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd791d-8b49-498c-900e-ae86c3be60d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:11.768603 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd791d-8b49-498c-900e-ae86c3be60d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:11.768914 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd791d-8b49-498c-900e-ae86c3be60d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:11.770293 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd791d-8b49-498c-900e-ae86c3be60d2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:11.789886 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:11.796239 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 360/730] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:11.798961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa0c1f-a647-4a3e-b68d-26422b4666bb None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:11.799251 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa0c1f-a647-4a3e-b68d-26422b4666bb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:11.799532 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa0c1f-a647-4a3e-b68d-26422b4666bb None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:11.800862 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 361/731] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:21:11.801605 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa0c1f-a647-4a3e-b68d-26422b4666bb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:11.869796 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5188ceb-56fd-4e4e-b9e6-0b9b71d56ca7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:11.873284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:11.918304 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8accb173-79c5-4ab8-94b9-7c4bc2259430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:11.919317 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accb173-79c5-4ab8-94b9-7c4bc2259430 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:11.919604 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accb173-79c5-4ab8-94b9-7c4bc2259430 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:11.919887 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8accb173-79c5-4ab8-94b9-7c4bc2259430 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:11.920869 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8accb173-79c5-4ab8-94b9-7c4bc2259430 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:11.921600 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8accb173-79c5-4ab8-94b9-7c4bc2259430 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:11.923596 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8accb173-79c5-4ab8-94b9-7c4bc2259430 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:11.928613 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa0c1f-a647-4a3e-b68d-26422b4666bb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:11.942092 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 371/732] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:11.943511 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec3aee-5d40-4335-9258-2c409b4ea8d0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:11.943809 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec3aee-5d40-4335-9258-2c409b4ea8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:11.944093 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec3aee-5d40-4335-9258-2c409b4ea8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:11.964007 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bcec3aee-5d40-4335-9258-2c409b4ea8d0 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:11.965871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bcec3aee-5d40-4335-9258-2c409b4ea8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:11.973052 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebe13d-c16a-47fa-8d0a-f2df95eda5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:11.973873 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebe13d-c16a-47fa-8d0a-f2df95eda5a1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:11.974163 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebe13d-c16a-47fa-8d0a-f2df95eda5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:11.974463 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebe13d-c16a-47fa-8d0a-f2df95eda5a1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:11.984472 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ebe13d-c16a-47fa-8d0a-f2df95eda5a1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:11.985923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ebe13d-c16a-47fa-8d0a-f2df95eda5a1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:12.003482 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e0ebe13d-c16a-47fa-8d0a-f2df95eda5a1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 12:21:12.003482 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:12.003482 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:12.003482 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:12.003482 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:12.003482 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:12.003482 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:12.003482 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:12.003482 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:12.003482 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:12.003482 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:12.003482 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:12.003482 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:21:12.003482 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:21:12.003482 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:21:12.003482 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:21:12.005336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:12.005336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:12.005336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:21:12.005336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:12.005336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:21:12.005336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:12.005336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:12.005336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:12.005336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:12.005336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:12.005336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:12.005336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:12.005336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:12.005336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:12.005336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:12.005336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:12.005336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:12.005336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:12.007584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:12.007584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:12.007584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:21:12.007584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:21:12.007584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:21:12.007584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:21:12.007584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:21:12.007584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:21:12.007584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 12:21:12.007584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:12.007584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 362/733] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jun 29 12:21:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:12.007584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebe13d-c16a-47fa-8d0a-f2df95eda5a1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:12.057539 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 372/734] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:21:12.058392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec3aee-5d40-4335-9258-2c409b4ea8d0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:12.078454 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:12.126491 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebe13d-c16a-47fa-8d0a-f2df95eda5a1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:12.164571 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362f046-5b5b-411c-a3f8-7ffd72e8f698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:12.165784 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362f046-5b5b-411c-a3f8-7ffd72e8f698 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:12.166214 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362f046-5b5b-411c-a3f8-7ffd72e8f698 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:12.166867 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362f046-5b5b-411c-a3f8-7ffd72e8f698 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:12.171170 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:12.175421 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8362f046-5b5b-411c-a3f8-7ffd72e8f698 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:12.177075 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8362f046-5b5b-411c-a3f8-7ffd72e8f698 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:12.181138 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec3aee-5d40-4335-9258-2c409b4ea8d0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:12.192523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8362f046-5b5b-411c-a3f8-7ffd72e8f698 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 12:21:12.192523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:12.192523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:12.192523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:12.192523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:12.192523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:12.192523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:12.192523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:12.192523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:12.192523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:12.192523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:12.192523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:12.192523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:21:12.192523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:21:12.192523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:21:12.192523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:21:12.193703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:12.193703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:12.193703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:21:12.193703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:12.193703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:21:12.193703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:12.193703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:12.193703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:12.193703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:12.193703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:12.193703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:12.193703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:12.193703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:12.193703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:12.193703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:12.193703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:12.193703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:12.193703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:12.194810 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:12.194810 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:12.194810 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:21:12.194810 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:21:12.194810 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:21:12.194810 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:21:12.194810 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:21:12.194810 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:21:12.194810 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 12:21:12.194810 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:12.196727 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 363/735] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:21:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:12.198150 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362f046-5b5b-411c-a3f8-7ffd72e8f698 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:12.218206 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e7c37-b388-48a7-8c05-66678ec73545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:12.218924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e7c37-b388-48a7-8c05-66678ec73545 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:12.219214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e7c37-b388-48a7-8c05-66678ec73545 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:12.219541 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e7c37-b388-48a7-8c05-66678ec73545 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:12.220392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44e7c37-b388-48a7-8c05-66678ec73545 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:12.220705 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44e7c37-b388-48a7-8c05-66678ec73545 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:12.222329 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44e7c37-b388-48a7-8c05-66678ec73545 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:12.241425 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 373/736] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Jun 29 12:21:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:12.242262 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e7c37-b388-48a7-8c05-66678ec73545 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:12.303213 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362f046-5b5b-411c-a3f8-7ffd72e8f698 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:12.324911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:12.341418 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfd09e-6958-4ccc-936e-bdcc85390103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:12.347743 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfd09e-6958-4ccc-936e-bdcc85390103 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:12.348254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfd09e-6958-4ccc-936e-bdcc85390103 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:12.348750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfd09e-6958-4ccc-936e-bdcc85390103 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:12.349691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcfd09e-6958-4ccc-936e-bdcc85390103 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:12.351360 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcfd09e-6958-4ccc-936e-bdcc85390103 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:12.351621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e7c37-b388-48a7-8c05-66678ec73545 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:12.382862 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 364/737] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jun 29 12:21:12 2022] POST /identity/v3/services => generated 251 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:21:12.384203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfd09e-6958-4ccc-936e-bdcc85390103 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:12.401843 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d92b6-6909-4e6c-b0ec-22cc642b3bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:12.402552 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d92b6-6909-4e6c-b0ec-22cc642b3bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:12.402842 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d92b6-6909-4e6c-b0ec-22cc642b3bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:12.403122 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024d92b6-6909-4e6c-b0ec-22cc642b3bc2 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:12.404433 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024d92b6-6909-4e6c-b0ec-22cc642b3bc2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:12.404740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024d92b6-6909-4e6c-b0ec-22cc642b3bc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:12.406394 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024d92b6-6909-4e6c-b0ec-22cc642b3bc2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:12.429446 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 374/738] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Jun 29 12:21:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:12.435498 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d92b6-6909-4e6c-b0ec-22cc642b3bc2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:12.448406 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:12.576684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfd09e-6958-4ccc-936e-bdcc85390103 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:12.610110 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024d92b6-6909-4e6c-b0ec-22cc642b3bc2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:12.619340 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e53fb-3085-454d-94cf-5dd66337ed3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:12.619930 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e53fb-3085-454d-94cf-5dd66337ed3a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:12.620336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e53fb-3085-454d-94cf-5dd66337ed3a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:12.620693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e53fb-3085-454d-94cf-5dd66337ed3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:12.653906 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b152e3a-fdec-40e1-8b85-44ccc35ba7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:12.654670 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b152e3a-fdec-40e1-8b85-44ccc35ba7ba None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:12.655045 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b152e3a-fdec-40e1-8b85-44ccc35ba7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:12.655437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b152e3a-fdec-40e1-8b85-44ccc35ba7ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:12.686812 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:12.720871 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e53fb-3085-454d-94cf-5dd66337ed3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:12.722375 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3e53fb-3085-454d-94cf-5dd66337ed3a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:12.749903 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b152e3a-fdec-40e1-8b85-44ccc35ba7ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:12.751396 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b152e3a-fdec-40e1-8b85-44ccc35ba7ba None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:12.766605 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:12.897730 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 375/739] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:21:12 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:21:12.903082 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b152e3a-fdec-40e1-8b85-44ccc35ba7ba None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:12.952392 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 365/740] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:21:12 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 29 12:21:12.959501 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e53fb-3085-454d-94cf-5dd66337ed3a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:13.089456 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b152e3a-fdec-40e1-8b85-44ccc35ba7ba None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:13.105036 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:13.110236 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:13.129895 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e53fb-3085-454d-94cf-5dd66337ed3a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:13.137726 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc65a6-ecb3-47f2-b25a-e832c6362caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:13.138672 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc65a6-ecb3-47f2-b25a-e832c6362caf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:13.138968 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc65a6-ecb3-47f2-b25a-e832c6362caf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:13.139255 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc65a6-ecb3-47f2-b25a-e832c6362caf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:13.151001 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dc65a6-ecb3-47f2-b25a-e832c6362caf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:13.152486 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dc65a6-ecb3-47f2-b25a-e832c6362caf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:13.167498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ba831-3c40-479b-9da4-deef65a6d941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:13.168292 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ba831-3c40-479b-9da4-deef65a6d941 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:13.168575 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ba831-3c40-479b-9da4-deef65a6d941 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:13.168853 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ba831-3c40-479b-9da4-deef65a6d941 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:13.171775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d3dc65a6-ecb3-47f2-b25a-e832c6362caf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:13.171775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:13.171775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:13.171775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:13.171775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:13.171775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:13.171775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:13.171775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:13.171775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:13.171775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:13.171775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:13.171775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:13.171775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:13.171775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:13.171775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:13.172635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:13.172635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:13.172635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:13.172635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:13.172635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:13.172635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:13.172635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:13.172635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:13.172635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:13.172635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:13.172635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:13.172635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:13.172635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:13.172635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:13.172635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:13.172635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:13.172635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:13.173539 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:13.173539 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:13.173539 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:13.173539 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:13.173539 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 376/741] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:13.174212 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da89252-9cba-4304-a165-08a61c9cbe08 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:13.178441 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da89252-9cba-4304-a165-08a61c9cbe08 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:13.178441 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da89252-9cba-4304-a165-08a61c9cbe08 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:13.178441 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 377/742] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:13.179715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91870aec-97af-408e-8152-14a8f5210457 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:13.180526 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91870aec-97af-408e-8152-14a8f5210457 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:13.180806 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91870aec-97af-408e-8152-14a8f5210457 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:13.182628 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ba831-3c40-479b-9da4-deef65a6d941 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:13.182974 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 378/743] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:13.184064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ba831-3c40-479b-9da4-deef65a6d941 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:13.187554 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91870aec-97af-408e-8152-14a8f5210457 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:13.201960 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e63ba831-3c40-479b-9da4-deef65a6d941 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:13.201960 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:13.201960 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:13.201960 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:13.201960 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:13.201960 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:13.201960 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:13.201960 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:13.201960 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:13.201960 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:13.201960 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:13.201960 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:13.201960 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:13.201960 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:13.201960 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:13.202986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:13.202986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:13.202986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:13.202986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:13.202986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:13.202986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:13.202986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:13.202986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:13.202986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:13.202986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:13.202986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:13.202986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:13.202986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:13.202986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:13.202986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:13.202986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:13.202986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:13.203888 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:13.203888 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:13.203888 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:13.203888 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:13.203888 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 366/744] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:13.204730 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc6388-8413-40db-815c-df495f7f9207 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:13.204730 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc6388-8413-40db-815c-df495f7f9207 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:13.204971 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc6388-8413-40db-815c-df495f7f9207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:13.226083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-65fc6388-8413-40db-815c-df495f7f9207 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:13.227330 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-65fc6388-8413-40db-815c-df495f7f9207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:13.300027 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91870aec-97af-408e-8152-14a8f5210457 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:13.311520 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 367/745] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:13 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:21:13.311520 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e6759-59a4-4a16-b7e5-20ce98648a14 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:13.311520 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e6759-59a4-4a16-b7e5-20ce98648a14 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:13.311520 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e6759-59a4-4a16-b7e5-20ce98648a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:13.330168 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-737e6759-59a4-4a16-b7e5-20ce98648a14 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:13.331423 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-737e6759-59a4-4a16-b7e5-20ce98648a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:13.336278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ca601-dcab-4763-bf6c-95d945520df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:13.337162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ca601-dcab-4763-bf6c-95d945520df8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:13.338194 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ca601-dcab-4763-bf6c-95d945520df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:13.338685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ca601-dcab-4763-bf6c-95d945520df8 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:13.339762 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6ca601-dcab-4763-bf6c-95d945520df8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:13.340257 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6ca601-dcab-4763-bf6c-95d945520df8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:13.341650 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6ca601-dcab-4763-bf6c-95d945520df8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:13.355770 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 379/746] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:13.356604 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ca601-dcab-4763-bf6c-95d945520df8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:13.378389 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:13.404671 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:13.418308 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 368/747] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:13 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 29 12:21:13.419049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e6759-59a4-4a16-b7e5-20ce98648a14 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:13.479247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ca601-dcab-4763-bf6c-95d945520df8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:13.514033 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b26f7-b62a-484b-bef3-a0d20f79db65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:13.514826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b26f7-b62a-484b-bef3-a0d20f79db65 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:13.515179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b26f7-b62a-484b-bef3-a0d20f79db65 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:13.515606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b26f7-b62a-484b-bef3-a0d20f79db65 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:13.516303 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b26f7-b62a-484b-bef3-a0d20f79db65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:13.516613 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b26f7-b62a-484b-bef3-a0d20f79db65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:13.517976 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b26f7-b62a-484b-bef3-a0d20f79db65 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:13.521968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e6759-59a4-4a16-b7e5-20ce98648a14 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:13.534271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 380/748] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:13.534271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b26f7-b62a-484b-bef3-a0d20f79db65 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:13.563603 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777389a4-2e5e-43a2-bbe3-fc043fae6422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:13.564574 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777389a4-2e5e-43a2-bbe3-fc043fae6422 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:13.565286 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777389a4-2e5e-43a2-bbe3-fc043fae6422 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:13.566119 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777389a4-2e5e-43a2-bbe3-fc043fae6422 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:13.567314 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777389a4-2e5e-43a2-bbe3-fc043fae6422 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:13.569134 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777389a4-2e5e-43a2-bbe3-fc043fae6422 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:13.586684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 369/749] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jun 29 12:21:13 2022] POST /identity/v3/services => generated 240 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:21:13.588961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777389a4-2e5e-43a2-bbe3-fc043fae6422 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:13.618994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:13.640530 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b26f7-b62a-484b-bef3-a0d20f79db65 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:13.671306 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:13.692379 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d0656-4288-4d7e-8cb9-6127af6783fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:13.693667 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d0656-4288-4d7e-8cb9-6127af6783fc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:13.693667 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d0656-4288-4d7e-8cb9-6127af6783fc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:13.693886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712d0656-4288-4d7e-8cb9-6127af6783fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:13.702575 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d0656-4288-4d7e-8cb9-6127af6783fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:13.704160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712d0656-4288-4d7e-8cb9-6127af6783fc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:13.721214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-712d0656-4288-4d7e-8cb9-6127af6783fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:13.721214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:13.721214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:13.721214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:13.721214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:13.721214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:13.721214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:13.721214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:13.721214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:13.721214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:13.721214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:13.721214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:13.721214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:21:13.721214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:21:13.721214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:21:13.722181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:21:13.722181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:13.722181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:13.722181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:13.722181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:13.722181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:13.722181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:13.722181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:13.722181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:13.722181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:13.722181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:13.722181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:13.722181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:13.722181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:13.722181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:13.722181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:13.722181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:13.723724 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:21:13.723724 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:21:13.723724 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:21:13.723724 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:21:13.723724 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:21:13.723724 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:21:13.723724 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:13.723724 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:13.723724 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 381/750] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:21:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:13.723724 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d0656-4288-4d7e-8cb9-6127af6783fc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:13.750800 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777389a4-2e5e-43a2-bbe3-fc043fae6422 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:13.801404 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca3050-848a-4e42-8638-e369f63be67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:13.802198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca3050-848a-4e42-8638-e369f63be67f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:13.802512 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca3050-848a-4e42-8638-e369f63be67f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:13.802787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca3050-848a-4e42-8638-e369f63be67f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:13.818239 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ca3050-848a-4e42-8638-e369f63be67f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:21:13.828521 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-70ca3050-848a-4e42-8638-e369f63be67f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:21:13.828521 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:13.828521 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:13.828521 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:13.828521 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:13.828521 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:13.828521 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:13.828521 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:13.828521 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:13.828521 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:13.828521 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:13.828521 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:13.828521 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:21:13.828521 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:21:13.828521 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:21:13.828521 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:21:13.829508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:21:13.829508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:21:13.829508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:21:13.829508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:21:13.829508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:13.829508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:13.829508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:13.829508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:13.829508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:13.829508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:13.829508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:13.829508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:13.829508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:13.829508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:13.829508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:13.829508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:13.830579 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:13.830579 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:13.830579 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:13.830579 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:13.830579 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:21:13.830579 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:21:13.830579 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:21:13.830579 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:21:13.830579 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:21:13.830579 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:21:13.830579 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:21:13.830579 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:13.830579 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 370/751] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:21:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:13.831432 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca3050-848a-4e42-8638-e369f63be67f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:13.847625 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712d0656-4288-4d7e-8cb9-6127af6783fc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:13.894275 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86d2d1-9ea2-4dee-b44d-e5a4b4357de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:13.895555 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86d2d1-9ea2-4dee-b44d-e5a4b4357de4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:13.895975 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86d2d1-9ea2-4dee-b44d-e5a4b4357de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:13.896636 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86d2d1-9ea2-4dee-b44d-e5a4b4357de4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:13.899214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:13.906191 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da86d2d1-9ea2-4dee-b44d-e5a4b4357de4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:13.908686 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da86d2d1-9ea2-4dee-b44d-e5a4b4357de4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:13.928247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-da86d2d1-9ea2-4dee-b44d-e5a4b4357de4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:13.928247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:13.928247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:13.928247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:13.928247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:13.928247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:13.928247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:13.928247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:13.928247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:13.928247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:13.928247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:13.928247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:13.928247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:21:13.928247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:21:13.928247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:21:13.929412 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:21:13.929412 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:13.929412 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:13.929412 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:13.929412 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:13.929412 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:13.929412 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:13.929412 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:13.929412 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:13.929412 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:13.929412 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:13.929412 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:13.929412 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:13.929412 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:13.929412 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:13.929412 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:13.929412 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:13.930561 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:21:13.930561 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:21:13.930561 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:21:13.930561 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:21:13.930561 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:21:13.930561 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:21:13.930561 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:13.930561 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:13.932963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 382/752] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:21:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:13.933877 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86d2d1-9ea2-4dee-b44d-e5a4b4357de4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:13.960316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca3050-848a-4e42-8638-e369f63be67f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:13.975760 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:14.004827 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f8a57-b62e-45c2-911c-ce686c265d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:14.005771 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f8a57-b62e-45c2-911c-ce686c265d2a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:14.006212 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f8a57-b62e-45c2-911c-ce686c265d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:14.006716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f8a57-b62e-45c2-911c-ce686c265d2a None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:14.007774 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f8a57-b62e-45c2-911c-ce686c265d2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:14.008342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f8a57-b62e-45c2-911c-ce686c265d2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:14.010228 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f8a57-b62e-45c2-911c-ce686c265d2a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:14.024352 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 371/753] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:21:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:14.025946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f8a57-b62e-45c2-911c-ce686c265d2a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:14.053522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86d2d1-9ea2-4dee-b44d-e5a4b4357de4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:14.090028 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f967a-3948-4d6f-8aeb-9badeace5407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:14.090874 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f967a-3948-4d6f-8aeb-9badeace5407 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:14.091255 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f967a-3948-4d6f-8aeb-9badeace5407 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:14.091631 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f967a-3948-4d6f-8aeb-9badeace5407 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:14.092470 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f967a-3948-4d6f-8aeb-9badeace5407 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:14.092884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f967a-3948-4d6f-8aeb-9badeace5407 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:14.094342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f967a-3948-4d6f-8aeb-9badeace5407 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:14.103139 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 383/754] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:21:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:14.104097 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f967a-3948-4d6f-8aeb-9badeace5407 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:14.140219 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f8a57-b62e-45c2-911c-ce686c265d2a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:14.165283 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:14.181193 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6287444-67c7-48ad-a556-bb07f278df44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:14.182267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6287444-67c7-48ad-a556-bb07f278df44 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:14.182824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6287444-67c7-48ad-a556-bb07f278df44 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:14.183286 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6287444-67c7-48ad-a556-bb07f278df44 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:14.184325 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6287444-67c7-48ad-a556-bb07f278df44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:14.184786 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6287444-67c7-48ad-a556-bb07f278df44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:14.186372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6287444-67c7-48ad-a556-bb07f278df44 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:14.200535 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 372/755] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:21:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:14.201732 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6287444-67c7-48ad-a556-bb07f278df44 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:14.220823 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f967a-3948-4d6f-8aeb-9badeace5407 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:14.237068 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:14.268824 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a997576-778f-4514-bfe7-8fd2222e71dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:14.269770 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a997576-778f-4514-bfe7-8fd2222e71dc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:14.270171 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a997576-778f-4514-bfe7-8fd2222e71dc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:14.270580 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a997576-778f-4514-bfe7-8fd2222e71dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:14.272420 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a997576-778f-4514-bfe7-8fd2222e71dc None admin] RBAC: Adding query filter params (role.id=3a7f4bf556344407a6d9540c89d13641, scope.project.id=f043ade9e4f748d4adc010ffb0a47a46, user.id=209b0df45f76404da880a9ae0c637d67) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:14.272837 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a997576-778f-4514-bfe7-8fd2222e71dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:14.274652 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a997576-778f-4514-bfe7-8fd2222e71dc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:14.283067 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 384/756] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:21:14 2022] GET /identity/v3/role_assignments?user.id=209b0df45f76404da880a9ae0c637d67&scope.project.id=f043ade9e4f748d4adc010ffb0a47a46&role.id=3a7f4bf556344407a6d9540c89d13641 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:14.284242 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a997576-778f-4514-bfe7-8fd2222e71dc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:14.326120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6287444-67c7-48ad-a556-bb07f278df44 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:14.389090 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70bc23d-c61b-469f-9f30-ee0a11aecaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:14.390379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bc23d-c61b-469f-9f30-ee0a11aecaa7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:14.391716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bc23d-c61b-469f-9f30-ee0a11aecaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:14.393949 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bc23d-c61b-469f-9f30-ee0a11aecaa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:14.420783 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70bc23d-c61b-469f-9f30-ee0a11aecaa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:14.423459 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70bc23d-c61b-469f-9f30-ee0a11aecaa7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:14.436565 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a997576-778f-4514-bfe7-8fd2222e71dc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:14.462451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c70bc23d-c61b-469f-9f30-ee0a11aecaa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:14.462451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:14.462451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:14.462451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:14.462451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:14.462451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:14.462451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:14.462451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:14.462451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:14.462451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:14.462451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:14.462451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:14.462451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:14.462451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:14.462451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:14.463981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:14.463981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:14.463981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:14.463981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:14.463981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:14.463981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:14.463981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:14.463981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:14.463981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:14.463981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:14.463981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:14.463981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:14.463981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:14.463981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:14.463981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:14.463981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:14.463981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:14.464893 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:14.464893 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:14.464893 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:14.464893 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:14.465882 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:14.470542 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 373/757] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:14.473124 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70bc23d-c61b-469f-9f30-ee0a11aecaa7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:14.488515 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d48eb-d1d3-4770-a167-1f12fde9a691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:14.489189 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d48eb-d1d3-4770-a167-1f12fde9a691 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:14.489471 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d48eb-d1d3-4770-a167-1f12fde9a691 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:14.489747 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1d48eb-d1d3-4770-a167-1f12fde9a691 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:14.490616 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d48eb-d1d3-4770-a167-1f12fde9a691 None admin] RBAC: Adding query filter params (role.id=8883c151055d4ba493f514ce1bb62e78, scope.project.id=f043ade9e4f748d4adc010ffb0a47a46, user.id=8c10852a728844c6ae0a48e3772d95b7) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:14.490931 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d48eb-d1d3-4770-a167-1f12fde9a691 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:14.492442 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1d48eb-d1d3-4770-a167-1f12fde9a691 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:14.499887 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 385/758] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:21:14 2022] GET /identity/v3/role_assignments?user.id=8c10852a728844c6ae0a48e3772d95b7&scope.project.id=f043ade9e4f748d4adc010ffb0a47a46&role.id=8883c151055d4ba493f514ce1bb62e78 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:14.601015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:14.628012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70bc23d-c61b-469f-9f30-ee0a11aecaa7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:14.674461 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96f94e8-5dbd-455f-9ddb-420406374426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:14.675552 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f94e8-5dbd-455f-9ddb-420406374426 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:14.676045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f94e8-5dbd-455f-9ddb-420406374426 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:14.676527 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96f94e8-5dbd-455f-9ddb-420406374426 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:14.677496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f94e8-5dbd-455f-9ddb-420406374426 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:14.678009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f94e8-5dbd-455f-9ddb-420406374426 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:14.679625 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96f94e8-5dbd-455f-9ddb-420406374426 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:14.694544 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 374/759] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:14.703620 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d48eb-d1d3-4770-a167-1f12fde9a691 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:14.710114 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605874a-c2a4-43e1-aa65-784952736df8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:14.710454 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605874a-c2a4-43e1-aa65-784952736df8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:14.710718 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605874a-c2a4-43e1-aa65-784952736df8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:14.712433 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 375/760] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:14.720623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b205f-f629-4f07-bc0d-806776b63bd3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:14.721243 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b205f-f629-4f07-bc0d-806776b63bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:14.721724 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b205f-f629-4f07-bc0d-806776b63bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:14.744643 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f48b205f-f629-4f07-bc0d-806776b63bd3 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:14.746215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f48b205f-f629-4f07-bc0d-806776b63bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:14.832313 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1d48eb-d1d3-4770-a167-1f12fde9a691 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:14.856981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 376/761] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:14.881963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7f52a-37bb-4731-9082-286d33e8c299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:14.883130 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7f52a-37bb-4731-9082-286d33e8c299 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:14.883524 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7f52a-37bb-4731-9082-286d33e8c299 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:14.883910 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd7f52a-37bb-4731-9082-286d33e8c299 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:14.899637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b205f-f629-4f07-bc0d-806776b63bd3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:14.903669 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd7f52a-37bb-4731-9082-286d33e8c299 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:14.910243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd7f52a-37bb-4731-9082-286d33e8c299 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:14.934118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-abd7f52a-37bb-4731-9082-286d33e8c299 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 12:21:14.934118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:14.934118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:14.934118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:14.934118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:14.934118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:14.934118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:14.934118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:14.934118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:14.934118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:14.934118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:14.934118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:14.934118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:21:14.934118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:21:14.934118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:21:14.934118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:21:14.935190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:14.935190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:14.935190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:21:14.935190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:14.935190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:21:14.935190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:14.935190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:14.935190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:14.935190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:14.935190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:14.935190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:14.935190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:14.935190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:14.935190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:14.935190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:14.935190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:14.935190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:14.935190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:14.936159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:14.936159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:14.936159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:21:14.936159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:21:14.936159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:21:14.936159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:21:14.936159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:21:14.936159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:21:14.936159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 12:21:14.936159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:14.940078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 386/762] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Jun 29 12:21:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:14.941754 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7f52a-37bb-4731-9082-286d33e8c299 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:15.006951 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:15.019515 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:15.042554 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b205f-f629-4f07-bc0d-806776b63bd3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:15.100326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd7f52a-37bb-4731-9082-286d33e8c299 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:15.125069 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91558575-0c86-44d0-97b6-81e7910c8d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:15.126211 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91558575-0c86-44d0-97b6-81e7910c8d95 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:15.126723 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91558575-0c86-44d0-97b6-81e7910c8d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:15.127210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91558575-0c86-44d0-97b6-81e7910c8d95 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:15.128072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91558575-0c86-44d0-97b6-81e7910c8d95 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:15.129661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91558575-0c86-44d0-97b6-81e7910c8d95 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:15.142267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-91558575-0c86-44d0-97b6-81e7910c8d95 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 12:21:15.142267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:15.142267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 12:21:15.142267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 12:21:15.142267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 12:21:15.142267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 12:21:15.142267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 12:21:15.142267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:15.142267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 12:21:15.142267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:15.142267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:21:15.142267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:15.142267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:15.142267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:15.142267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:15.142267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:15.143679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:15.143679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:15.143679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:15.143679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:15.143679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:15.143679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:15.143679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:15.143679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 12:21:15.143679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 12:21:15.143679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 12:21:15.143679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 12:21:15.143679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:15.143679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:15.143679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:15.143679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:15.143679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:15.144529 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:15.144529 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:15.144529 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:15.144529 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:15.144529 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:15.144529 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:15.144529 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:15.144529 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:15.144529 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:15.144529 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:15.144529 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:15.144529 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 12:21:15.144529 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:15.144529 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 12:21:15.144529 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:15.147715 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 377/763] 10.222.0.52 () {56 vars in 1129 bytes} [Wed Jun 29 12:21:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:15.154253 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91558575-0c86-44d0-97b6-81e7910c8d95 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:15.167261 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c7568-c770-4d12-b39a-3ab102ed4948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:15.168004 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c7568-c770-4d12-b39a-3ab102ed4948 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:15.168326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c7568-c770-4d12-b39a-3ab102ed4948 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:15.168737 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c7568-c770-4d12-b39a-3ab102ed4948 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:15.169596 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c7568-c770-4d12-b39a-3ab102ed4948 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:15.169912 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c7568-c770-4d12-b39a-3ab102ed4948 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:15.171506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c7568-c770-4d12-b39a-3ab102ed4948 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:15.196610 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 387/764] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Jun 29 12:21:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:15.202099 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c7568-c770-4d12-b39a-3ab102ed4948 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:15.308607 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c7568-c770-4d12-b39a-3ab102ed4948 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:15.310244 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:15.343183 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91558575-0c86-44d0-97b6-81e7910c8d95 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:15.389300 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b382743-f586-495a-8b6f-39f75553c6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:15.390449 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b382743-f586-495a-8b6f-39f75553c6f4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:15.390933 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b382743-f586-495a-8b6f-39f75553c6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:15.391403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b382743-f586-495a-8b6f-39f75553c6f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:15.410081 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b382743-f586-495a-8b6f-39f75553c6f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:15.412702 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b382743-f586-495a-8b6f-39f75553c6f4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:15.416325 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141d091-8ecc-487e-8f62-3b1dc47c9a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:15.418317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141d091-8ecc-487e-8f62-3b1dc47c9a58 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:15.419044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141d091-8ecc-487e-8f62-3b1dc47c9a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:15.420333 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141d091-8ecc-487e-8f62-3b1dc47c9a58 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:15.425293 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141d091-8ecc-487e-8f62-3b1dc47c9a58 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:15.425839 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141d091-8ecc-487e-8f62-3b1dc47c9a58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:15.427428 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141d091-8ecc-487e-8f62-3b1dc47c9a58 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:15.434578 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 378/765] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Jun 29 12:21:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:15.440252 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141d091-8ecc-487e-8f62-3b1dc47c9a58 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:15.445782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-0b382743-f586-495a-8b6f-39f75553c6f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:15.445782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:15.445782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:15.445782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:15.445782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:15.445782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:15.445782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:15.445782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:15.445782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:15.445782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:15.445782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:15.445782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:15.445782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:15.445782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:15.445782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:15.447164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:15.447164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:15.447164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:15.447164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:15.447164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:15.447164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:15.447164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:15.447164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:15.447164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:15.447164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:15.447164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:15.447164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:15.447164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:15.447164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:15.447164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:15.447164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:15.447164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:15.448134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:15.448134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:15.448134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:15.448134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:15.456147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 388/766] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:15.460501 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:15.461246 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b382743-f586-495a-8b6f-39f75553c6f4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:15.595775 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141d091-8ecc-487e-8f62-3b1dc47c9a58 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:15.648685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b382743-f586-495a-8b6f-39f75553c6f4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:15.661196 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849786d-a2bd-4de1-9061-31466d6d9a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:15.662158 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849786d-a2bd-4de1-9061-31466d6d9a1d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:15.662761 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849786d-a2bd-4de1-9061-31466d6d9a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:15.664371 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849786d-a2bd-4de1-9061-31466d6d9a1d None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:15.666597 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849786d-a2bd-4de1-9061-31466d6d9a1d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:15.667261 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849786d-a2bd-4de1-9061-31466d6d9a1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:15.668933 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849786d-a2bd-4de1-9061-31466d6d9a1d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:15.679681 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 379/767] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Jun 29 12:21:15 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:15.685012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849786d-a2bd-4de1-9061-31466d6d9a1d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:15.720841 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e5718-a0a9-42f2-9433-0c03c2dc2458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:15.721950 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e5718-a0a9-42f2-9433-0c03c2dc2458 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:15.722448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e5718-a0a9-42f2-9433-0c03c2dc2458 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:15.722921 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e5718-a0a9-42f2-9433-0c03c2dc2458 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:15.723881 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44e5718-a0a9-42f2-9433-0c03c2dc2458 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:15.727201 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44e5718-a0a9-42f2-9433-0c03c2dc2458 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:15.728829 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44e5718-a0a9-42f2-9433-0c03c2dc2458 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:15.750103 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 389/768] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:15.754697 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e5718-a0a9-42f2-9433-0c03c2dc2458 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:15.774202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:15.807725 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:15.852032 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849786d-a2bd-4de1-9061-31466d6d9a1d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:15.881865 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e5718-a0a9-42f2-9433-0c03c2dc2458 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:15.895133 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0707c8-7ca1-452f-8199-112e03ae7b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:15.895822 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0707c8-7ca1-452f-8199-112e03ae7b44 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:15.896115 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0707c8-7ca1-452f-8199-112e03ae7b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:15.896669 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0707c8-7ca1-452f-8199-112e03ae7b44 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:15.897479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0707c8-7ca1-452f-8199-112e03ae7b44 None admin] RBAC: Adding query filter params (interface=public, service_id=56444d85407e47ada73b37aba981fa54, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:15.897811 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0707c8-7ca1-452f-8199-112e03ae7b44 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:15.899267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0707c8-7ca1-452f-8199-112e03ae7b44 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:15.914255 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 380/769] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Jun 29 12:21:15 2022] GET /identity/v3/endpoints?service_id=56444d85407e47ada73b37aba981fa54&interface=public®ion_id=RegionOne => generated 194 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:15.921212 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0707c8-7ca1-452f-8199-112e03ae7b44 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:15.937858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2456f-452d-41b4-a1eb-a3c52342d4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:15.939031 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2456f-452d-41b4-a1eb-a3c52342d4cd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:15.939922 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2456f-452d-41b4-a1eb-a3c52342d4cd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:15.941513 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2456f-452d-41b4-a1eb-a3c52342d4cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:15.953731 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2456f-452d-41b4-a1eb-a3c52342d4cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:15.955708 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2456f-452d-41b4-a1eb-a3c52342d4cd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:15.973727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-55d2456f-452d-41b4-a1eb-a3c52342d4cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:15.973727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:15.973727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:15.973727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:15.973727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:15.973727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:15.973727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:15.973727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:15.973727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:15.973727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:15.973727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:15.973727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:15.973727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:21:15.973727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:21:15.973727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:21:15.974921 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:21:15.974921 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:15.974921 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:15.974921 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:15.974921 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:15.974921 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:15.974921 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:15.974921 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:15.974921 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:15.974921 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:15.974921 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:15.974921 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:15.974921 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:15.974921 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:15.974921 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:15.974921 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:15.974921 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:15.976242 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:21:15.976242 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:21:15.976242 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:21:15.976242 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:21:15.976242 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:21:15.976242 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:21:15.976242 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:15.976242 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:15.976242 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 390/770] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:21:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:15.976726 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722a8b7-02a6-45f6-9f6d-67434eb14331 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:15.976950 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722a8b7-02a6-45f6-9f6d-67434eb14331 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:15.977123 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722a8b7-02a6-45f6-9f6d-67434eb14331 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:15.979618 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 391/771] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:15.983460 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5722a8b7-02a6-45f6-9f6d-67434eb14331 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:16.061571 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0707c8-7ca1-452f-8199-112e03ae7b44 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:16.083846 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:16.102567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:16.114574 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b39d7-1d57-41ee-bc06-d7efaf19210d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:16.115256 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b39d7-1d57-41ee-bc06-d7efaf19210d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:16.115546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b39d7-1d57-41ee-bc06-d7efaf19210d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:16.115827 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b39d7-1d57-41ee-bc06-d7efaf19210d None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:16.116514 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b39d7-1d57-41ee-bc06-d7efaf19210d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:16.116839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5722a8b7-02a6-45f6-9f6d-67434eb14331 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:16.117952 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b39d7-1d57-41ee-bc06-d7efaf19210d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:16.130821 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 381/772] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Jun 29 12:21:15 2022] GET /identity/v3/services => generated 772 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:16.132259 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672915f-3c75-44ce-be61-3123e8f7fe3a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:16.132557 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672915f-3c75-44ce-be61-3123e8f7fe3a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:16.132840 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672915f-3c75-44ce-be61-3123e8f7fe3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:16.158648 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5aecd-3e65-4471-a826-d980f05a09e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:16.159566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5aecd-3e65-4471-a826-d980f05a09e0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:16.160225 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5aecd-3e65-4471-a826-d980f05a09e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:16.160764 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e5aecd-3e65-4471-a826-d980f05a09e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:16.161705 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e5aecd-3e65-4471-a826-d980f05a09e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:16.162160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e5aecd-3e65-4471-a826-d980f05a09e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:16.163933 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e5aecd-3e65-4471-a826-d980f05a09e0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:16.167658 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e672915f-3c75-44ce-be61-3123e8f7fe3a None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:16.168946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e672915f-3c75-44ce-be61-3123e8f7fe3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:16.179540 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 392/773] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:21:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:16.186495 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5aecd-3e65-4471-a826-d980f05a09e0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:16.338915 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e5aecd-3e65-4471-a826-d980f05a09e0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:16.350805 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 382/774] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:16.374399 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672915f-3c75-44ce-be61-3123e8f7fe3a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:16.397691 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a7af6-074b-4a0d-8758-5a87f6dfcad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:16.398877 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a7af6-074b-4a0d-8758-5a87f6dfcad2 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:16.399303 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a7af6-074b-4a0d-8758-5a87f6dfcad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:16.399723 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7a7af6-074b-4a0d-8758-5a87f6dfcad2 None admin] PATH_INFO: `/v3/projects/f043ade9e4f748d4adc010ffb0a47a46/users/3ac7391a3c41499d94e5f97478797313/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:16.402553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7a7af6-074b-4a0d-8758-5a87f6dfcad2 None admin] RBAC: Authorizing `identity:create_grant(project_id=f043ade9e4f748d4adc010ffb0a47a46, user_id=3ac7391a3c41499d94e5f97478797313, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:16.404282 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7a7af6-074b-4a0d-8758-5a87f6dfcad2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:16.407025 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:16.428725 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 393/775] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 12:21:16 2022] PUT /identity/v3/projects/f043ade9e4f748d4adc010ffb0a47a46/users/3ac7391a3c41499d94e5f97478797313/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:21:16.538864 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672915f-3c75-44ce-be61-3123e8f7fe3a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:16.575508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:16.591752 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380a8fe-a9a5-40dc-a210-111ad584684c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:16.592574 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380a8fe-a9a5-40dc-a210-111ad584684c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:16.592866 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380a8fe-a9a5-40dc-a210-111ad584684c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:16.593150 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380a8fe-a9a5-40dc-a210-111ad584684c None admin] PATH_INFO: `/v3/services/network` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:16.593820 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380a8fe-a9a5-40dc-a210-111ad584684c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:16.595236 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380a8fe-a9a5-40dc-a210-111ad584684c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:16.611656 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9380a8fe-a9a5-40dc-a210-111ad584684c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 12:21:16.611656 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:16.611656 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 12:21:16.611656 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 12:21:16.611656 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 12:21:16.611656 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 12:21:16.611656 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 12:21:16.611656 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:16.611656 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 12:21:16.611656 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:16.611656 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:21:16.611656 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:16.611656 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:16.611656 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:16.611656 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:16.611656 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:16.611656 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:16.612728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:16.612728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:16.612728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:16.612728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:16.612728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:16.612728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:16.612728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 12:21:16.612728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 12:21:16.612728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 12:21:16.612728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 12:21:16.612728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:16.612728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:16.612728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:16.612728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:16.612728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:16.612728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:16.613978 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:16.613978 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:16.613978 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:16.613978 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:16.613978 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:16.613978 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:16.613978 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:16.613978 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:16.613978 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:16.613978 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:16.613978 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 12:21:16.613978 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:16.613978 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 12:21:16.613978 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:16.613978 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 383/776] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:21:16 2022] GET /identity/v3/services/network => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:16.619428 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a7af6-074b-4a0d-8758-5a87f6dfcad2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:16.741973 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7a7af6-074b-4a0d-8758-5a87f6dfcad2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:16.771568 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567b5a5-4803-4347-ae13-85858726b081 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:16.771904 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567b5a5-4803-4347-ae13-85858726b081 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:16.772158 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567b5a5-4803-4347-ae13-85858726b081 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:16.773839 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 384/777] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:21:16.780135 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef0b8e8-7040-49c5-ac2c-83e282eea5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:16.780807 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0b8e8-7040-49c5-ac2c-83e282eea5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:16.781088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0b8e8-7040-49c5-ac2c-83e282eea5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:16.781364 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0b8e8-7040-49c5-ac2c-83e282eea5d6 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:16.781759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9128f55-0642-4324-a76c-ffc4e2d5dc77 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:16.782073 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef0b8e8-7040-49c5-ac2c-83e282eea5d6 None admin] RBAC: Adding query filter params (name=network) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:16.782382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef0b8e8-7040-49c5-ac2c-83e282eea5d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:16.782674 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9128f55-0642-4324-a76c-ffc4e2d5dc77 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:16.783178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9128f55-0642-4324-a76c-ffc4e2d5dc77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:16.783799 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef0b8e8-7040-49c5-ac2c-83e282eea5d6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:16.789352 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 394/778] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jun 29 12:21:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:16.798457 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef0b8e8-7040-49c5-ac2c-83e282eea5d6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:16.816423 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f9128f55-0642-4324-a76c-ffc4e2d5dc77 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:16.818533 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f9128f55-0642-4324-a76c-ffc4e2d5dc77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:16.909369 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:16.954971 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef0b8e8-7040-49c5-ac2c-83e282eea5d6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:16.976624 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 385/779] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 12:21:17.007248 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ec858-2002-4037-bb37-ed2227f34633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:17.007941 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ec858-2002-4037-bb37-ed2227f34633 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:17.008223 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ec858-2002-4037-bb37-ed2227f34633 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:17.008497 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205ec858-2002-4037-bb37-ed2227f34633 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:17.009982 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ec858-2002-4037-bb37-ed2227f34633 None admin] RBAC: Adding query filter params (type=network) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:17.010300 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ec858-2002-4037-bb37-ed2227f34633 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:17.011726 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205ec858-2002-4037-bb37-ed2227f34633 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:17.019358 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 395/780] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jun 29 12:21:16 2022] GET /identity/v3/services?type=network => generated 350 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:17.026471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9128f55-0642-4324-a76c-ffc4e2d5dc77 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:17.034918 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ec858-2002-4037-bb37-ed2227f34633 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:17.203433 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205ec858-2002-4037-bb37-ed2227f34633 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:17.234453 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:17.248807 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9128f55-0642-4324-a76c-ffc4e2d5dc77 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:17.250089 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5724d-2353-44df-8a4a-5ebe1787f7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:17.250923 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5724d-2353-44df-8a4a-5ebe1787f7dd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:17.251503 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5724d-2353-44df-8a4a-5ebe1787f7dd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:17.251840 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5724d-2353-44df-8a4a-5ebe1787f7dd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:17.253630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5724d-2353-44df-8a4a-5ebe1787f7dd None admin] RBAC: Adding query filter params (interface=public, service_id=78fa29cffc2f44f887352bc6fccfe8b2, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:17.253958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5724d-2353-44df-8a4a-5ebe1787f7dd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:17.255382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c5724d-2353-44df-8a4a-5ebe1787f7dd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:17.265321 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 396/781] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Jun 29 12:21:17 2022] GET /identity/v3/endpoints?service_id=78fa29cffc2f44f887352bc6fccfe8b2&interface=public®ion_id=RegionOne => generated 194 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:17.266691 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f707d56-fa2a-4cd2-a0f6-0aa1b38c2fa6 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:17.266979 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f707d56-fa2a-4cd2-a0f6-0aa1b38c2fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:17.267260 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f707d56-fa2a-4cd2-a0f6-0aa1b38c2fa6 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:17.268805 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 397/782] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:17.272518 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f707d56-fa2a-4cd2-a0f6-0aa1b38c2fa6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:17.288984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:17.313825 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0c2e1-af74-4136-89da-a1951563db14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:17.314678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0c2e1-af74-4136-89da-a1951563db14 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:17.314964 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0c2e1-af74-4136-89da-a1951563db14 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:17.315242 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0c2e1-af74-4136-89da-a1951563db14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:17.327582 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb0c2e1-af74-4136-89da-a1951563db14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:21:17.335555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-feb0c2e1-af74-4136-89da-a1951563db14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:21:17.335555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:17.335555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:17.335555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:17.335555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:17.335555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:17.335555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:17.335555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:17.335555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:17.335555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:17.335555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:17.335555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:17.335555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:21:17.335555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:21:17.335555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:21:17.335555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:21:17.336449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:21:17.336449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:21:17.336449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:21:17.336449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:21:17.336449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:17.336449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:17.336449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:17.336449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:17.336449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:17.336449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:17.336449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:17.336449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:17.336449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:17.336449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:17.336449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:17.336449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:17.337285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:17.337285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:17.337285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:17.337285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:17.337285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:21:17.337285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:21:17.337285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:21:17.337285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:21:17.337285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:21:17.337285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:21:17.337285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:21:17.337285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:17.337285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 386/783] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:21:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:17.338330 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147d1c9-75fd-4493-a843-f2a4f441f069 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:17.338771 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147d1c9-75fd-4493-a843-f2a4f441f069 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:17.350074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5147d1c9-75fd-4493-a843-f2a4f441f069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:17.374944 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5147d1c9-75fd-4493-a843-f2a4f441f069 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:17.376499 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5147d1c9-75fd-4493-a843-f2a4f441f069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:17.403912 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f707d56-fa2a-4cd2-a0f6-0aa1b38c2fa6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:17.445666 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f7720-5304-4373-9597-214fd4e3eee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:17.446347 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f7720-5304-4373-9597-214fd4e3eee5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:17.446679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f7720-5304-4373-9597-214fd4e3eee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:17.447134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f7720-5304-4373-9597-214fd4e3eee5 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:17.447824 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255f7720-5304-4373-9597-214fd4e3eee5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:17.450928 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255f7720-5304-4373-9597-214fd4e3eee5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:17.458334 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 398/784] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Jun 29 12:21:17 2022] GET /identity/v3/services => generated 772 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:17.459458 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f7720-5304-4373-9597-214fd4e3eee5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:17.460770 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 387/785] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:17.512510 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5147d1c9-75fd-4493-a843-f2a4f441f069 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:17.528253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:17.586632 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f7720-5304-4373-9597-214fd4e3eee5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:17.630083 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5a959-caa0-4a6a-89c5-045c54367ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:17.631751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5a959-caa0-4a6a-89c5-045c54367ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:17.632024 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5a959-caa0-4a6a-89c5-045c54367ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:17.632301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5a959-caa0-4a6a-89c5-045c54367ef4 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:17.633082 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5a959-caa0-4a6a-89c5-045c54367ef4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:17.633392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5a959-caa0-4a6a-89c5-045c54367ef4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:17.635675 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5a959-caa0-4a6a-89c5-045c54367ef4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:17.657356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 399/786] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:21:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:17.658703 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5a959-caa0-4a6a-89c5-045c54367ef4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:17.706649 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5147d1c9-75fd-4493-a843-f2a4f441f069 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:17.717417 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:17.747222 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f48bdd2-5897-4deb-b965-cabed6508b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:17.748043 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48bdd2-5897-4deb-b965-cabed6508b5f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:17.748325 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48bdd2-5897-4deb-b965-cabed6508b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:17.748598 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48bdd2-5897-4deb-b965-cabed6508b5f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:17.764560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f48bdd2-5897-4deb-b965-cabed6508b5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:21:17.774217 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5f48bdd2-5897-4deb-b965-cabed6508b5f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:21:17.774217 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:17.774217 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:17.774217 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:17.774217 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:17.774217 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:17.774217 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:17.774217 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:17.774217 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:17.774217 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:17.774217 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:17.774217 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:17.774217 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:21:17.774217 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:21:17.774217 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:21:17.774217 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:21:17.775352 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:21:17.775352 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:21:17.775352 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:21:17.775352 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:21:17.775352 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:17.775352 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:17.775352 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:17.775352 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:17.775352 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:17.775352 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:17.775352 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:17.775352 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:17.775352 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:17.775352 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:17.775352 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:17.775352 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:17.776436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:17.776436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:17.776436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:17.776436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:17.776436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:21:17.776436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:21:17.776436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:21:17.776436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:21:17.776436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:21:17.776436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:21:17.776436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:21:17.776436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:17.776436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 388/787] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 12:21:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:17.786304 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f48bdd2-5897-4deb-b965-cabed6508b5f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:17.795873 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5a959-caa0-4a6a-89c5-045c54367ef4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:17.814021 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:17.843179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03f165-7f9d-4373-ab6e-bc1d19028b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:17.844658 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03f165-7f9d-4373-ab6e-bc1d19028b70 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:17.845089 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03f165-7f9d-4373-ab6e-bc1d19028b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:17.845884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc03f165-7f9d-4373-ab6e-bc1d19028b70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:17.860952 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03f165-7f9d-4373-ab6e-bc1d19028b70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:17.863289 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc03f165-7f9d-4373-ab6e-bc1d19028b70 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:17.888063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-bc03f165-7f9d-4373-ab6e-bc1d19028b70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:17.888063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:17.888063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:17.888063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:17.888063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:17.888063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:17.888063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:17.888063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:17.888063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:17.888063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:17.888063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:17.888063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:17.888063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:17.888063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:17.888063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:17.889179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:17.889179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:17.889179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:17.889179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:17.889179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:17.889179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:17.889179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:17.889179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:17.889179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:17.889179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:17.889179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:17.889179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:17.889179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:17.889179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:17.889179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:17.889179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:17.889179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:17.890065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:17.890065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:17.890065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:17.890065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:17.897502 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 400/788] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:17.900035 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03f165-7f9d-4373-ab6e-bc1d19028b70 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:17.918583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f48bdd2-5897-4deb-b965-cabed6508b5f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:17.988527 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2f419-b537-48b2-a35f-24d1de1b8643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:17.989207 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2f419-b537-48b2-a35f-24d1de1b8643 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:17.989489 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2f419-b537-48b2-a35f-24d1de1b8643 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:17.989767 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf2f419-b537-48b2-a35f-24d1de1b8643 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:17.990547 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf2f419-b537-48b2-a35f-24d1de1b8643 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:17.995165 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf2f419-b537-48b2-a35f-24d1de1b8643 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:17.997575 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf2f419-b537-48b2-a35f-24d1de1b8643 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:18.016410 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 389/789] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 12:21:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:18.020395 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2f419-b537-48b2-a35f-24d1de1b8643 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:18.070992 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc03f165-7f9d-4373-ab6e-bc1d19028b70 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:18.100264 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:18.115413 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4294f44b-250e-44ad-a15d-27a8627f7f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:18.116094 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294f44b-250e-44ad-a15d-27a8627f7f22 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:18.116384 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294f44b-250e-44ad-a15d-27a8627f7f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:18.116659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294f44b-250e-44ad-a15d-27a8627f7f22 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:18.117389 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294f44b-250e-44ad-a15d-27a8627f7f22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:18.117694 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294f44b-250e-44ad-a15d-27a8627f7f22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:18.119645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294f44b-250e-44ad-a15d-27a8627f7f22 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:18.141870 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 401/790] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:18.149453 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4294f44b-250e-44ad-a15d-27a8627f7f22 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:18.180713 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf2f419-b537-48b2-a35f-24d1de1b8643 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:18.201784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:18.222575 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874dd6df-c071-4a51-a2c4-f869cd59259d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:18.223641 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dd6df-c071-4a51-a2c4-f869cd59259d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:18.224106 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dd6df-c071-4a51-a2c4-f869cd59259d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:18.224563 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dd6df-c071-4a51-a2c4-f869cd59259d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:18.237167 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874dd6df-c071-4a51-a2c4-f869cd59259d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:18.238906 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874dd6df-c071-4a51-a2c4-f869cd59259d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:18.265646 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-874dd6df-c071-4a51-a2c4-f869cd59259d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:18.265646 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:18.265646 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:18.265646 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:18.265646 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:18.265646 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:18.265646 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:18.265646 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:18.265646 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:18.265646 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:18.265646 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:18.265646 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:18.265646 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:18.265646 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:18.265646 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:18.267034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:18.267034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:18.267034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:18.267034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:18.267034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:18.267034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:18.267034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:18.267034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:18.267034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:18.267034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:18.267034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:18.267034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:18.267034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:18.267034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:18.267034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:18.267034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:18.267034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:18.267928 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:18.267928 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:18.267928 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:18.267928 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:18.268159 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 390/791] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:18.276640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874dd6df-c071-4a51-a2c4-f869cd59259d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:18.337976 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4294f44b-250e-44ad-a15d-27a8627f7f22 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:18.411718 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c4f6a9-c259-4cab-85c5-89e7113fab8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:18.412524 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4f6a9-c259-4cab-85c5-89e7113fab8a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:18.412805 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4f6a9-c259-4cab-85c5-89e7113fab8a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:18.413082 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c4f6a9-c259-4cab-85c5-89e7113fab8a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:18.426300 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c4f6a9-c259-4cab-85c5-89e7113fab8a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:18.427809 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c4f6a9-c259-4cab-85c5-89e7113fab8a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:18.452280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-21c4f6a9-c259-4cab-85c5-89e7113fab8a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 12:21:18.452280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:18.452280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:18.452280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:18.452280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:18.452280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:18.452280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:18.452280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:18.452280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:18.452280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:18.452280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:18.452280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:18.452280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:21:18.452280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:21:18.452280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:21:18.452280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:21:18.453291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:18.453291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:18.453291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:21:18.453291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:18.453291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:21:18.453291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:18.453291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:18.453291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:18.453291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:18.453291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:18.453291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:18.453291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:18.453291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:18.453291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:18.453291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:18.453291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:18.453291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:18.453291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:18.454228 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:18.454228 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:18.454228 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:21:18.454228 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:21:18.454228 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:21:18.454228 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:21:18.454228 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:21:18.454228 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:21:18.454228 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 12:21:18.454228 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:18.454228 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 402/792] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:21:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:18.457306 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:18.460917 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c4f6a9-c259-4cab-85c5-89e7113fab8a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:18.495741 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874dd6df-c071-4a51-a2c4-f869cd59259d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:18.556596 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a65fa5-aa0c-40b4-af49-73c93e101993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:18.557439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a65fa5-aa0c-40b4-af49-73c93e101993 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:18.557722 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a65fa5-aa0c-40b4-af49-73c93e101993 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:18.557996 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a65fa5-aa0c-40b4-af49-73c93e101993 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:18.558751 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a65fa5-aa0c-40b4-af49-73c93e101993 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:18.559064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a65fa5-aa0c-40b4-af49-73c93e101993 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:18.560450 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a65fa5-aa0c-40b4-af49-73c93e101993 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:18.579012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 391/793] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:18.584859 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a65fa5-aa0c-40b4-af49-73c93e101993 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:18.602878 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:18.629105 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c4f6a9-c259-4cab-85c5-89e7113fab8a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:18.701722 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d1c4f-f063-4dd3-aefe-0503221210b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:18.702520 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d1c4f-f063-4dd3-aefe-0503221210b6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:18.702719 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d1c4f-f063-4dd3-aefe-0503221210b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:18.703094 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d1c4f-f063-4dd3-aefe-0503221210b6 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:18.703956 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d1c4f-f063-4dd3-aefe-0503221210b6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:18.704274 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d1c4f-f063-4dd3-aefe-0503221210b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:18.705726 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d1c4f-f063-4dd3-aefe-0503221210b6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:18.742556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 403/794] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Jun 29 12:21:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:18.746273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a65fa5-aa0c-40b4-af49-73c93e101993 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:18.749920 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d1c4f-f063-4dd3-aefe-0503221210b6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:18.823358 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a148544-36a9-4390-9abc-136191281b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:18.824106 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a148544-36a9-4390-9abc-136191281b8b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:18.824391 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a148544-36a9-4390-9abc-136191281b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:18.824665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a148544-36a9-4390-9abc-136191281b8b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:18.832347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a148544-36a9-4390-9abc-136191281b8b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:18.833866 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a148544-36a9-4390-9abc-136191281b8b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:18.859035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2a148544-36a9-4390-9abc-136191281b8b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 12:21:18.859035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:18.859035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:18.859035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:18.859035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:18.859035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:18.859035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:18.859035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:18.859035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:18.859035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:18.859035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:18.859035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:18.859035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:21:18.859035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:21:18.859035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:21:18.859035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:21:18.860020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:18.860020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:18.860020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:21:18.860020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:18.860020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:21:18.860020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:18.860020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:18.860020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:18.860020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:18.860020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:18.860020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:18.860020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:18.860020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:18.860020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:18.860020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:18.860020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:18.860020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:18.860020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:18.860947 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:18.860947 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:18.860947 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:21:18.860947 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:21:18.860947 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:21:18.860947 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:21:18.860947 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:21:18.860947 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:21:18.860947 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 12:21:18.860947 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:18.865000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 392/795] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jun 29 12:21:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:18.866338 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a148544-36a9-4390-9abc-136191281b8b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:18.870789 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:18.883771 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d1c4f-f063-4dd3-aefe-0503221210b6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:18.921262 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfeda83-9447-4c42-a5ab-820bd56b9a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:18.922249 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfeda83-9447-4c42-a5ab-820bd56b9a60 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:18.922711 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfeda83-9447-4c42-a5ab-820bd56b9a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:18.923094 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfeda83-9447-4c42-a5ab-820bd56b9a60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:18.933812 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfeda83-9447-4c42-a5ab-820bd56b9a60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:18.935455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfeda83-9447-4c42-a5ab-820bd56b9a60 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:18.942942 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:18.955735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-8dfeda83-9447-4c42-a5ab-820bd56b9a60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:18.955735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:18.955735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:18.955735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:18.955735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:18.955735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:18.955735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:18.955735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:18.955735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:18.955735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:18.955735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:18.955735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:18.955735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:18.955735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:18.955735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:18.956755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:18.956755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:18.956755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:18.956755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:18.956755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:18.956755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:18.956755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:18.956755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:18.956755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:18.956755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:18.956755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:18.956755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:18.956755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:18.956755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:18.956755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:18.956755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:18.956755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:18.957648 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:18.957648 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:18.957648 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:18.957648 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:18.958991 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 404/796] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:18.963992 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfeda83-9447-4c42-a5ab-820bd56b9a60 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:18.998092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a148544-36a9-4390-9abc-136191281b8b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:19.034096 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8cb2b-c04c-4efb-8735-b2d61ff969d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:19.034847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8cb2b-c04c-4efb-8735-b2d61ff969d7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:19.035137 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8cb2b-c04c-4efb-8735-b2d61ff969d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:19.035418 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8cb2b-c04c-4efb-8735-b2d61ff969d7 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:19.036311 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d8cb2b-c04c-4efb-8735-b2d61ff969d7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:19.036731 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d8cb2b-c04c-4efb-8735-b2d61ff969d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:19.038356 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d8cb2b-c04c-4efb-8735-b2d61ff969d7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:19.057302 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 393/797] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Jun 29 12:21:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:19.058659 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c2137-c6a3-4931-8bc2-2e62e3b776ee None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:19.058956 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c2137-c6a3-4931-8bc2-2e62e3b776ee None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:19.059237 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2c2137-c6a3-4931-8bc2-2e62e3b776ee None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:19.064813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 394/798] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:19.065701 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c2137-c6a3-4931-8bc2-2e62e3b776ee None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:19.074902 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfeda83-9447-4c42-a5ab-820bd56b9a60 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:19.109818 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1072451-5e50-45f7-b9a0-cd0a7404a6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:19.110398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1072451-5e50-45f7-b9a0-cd0a7404a6b4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:19.110715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1072451-5e50-45f7-b9a0-cd0a7404a6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:19.110999 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1072451-5e50-45f7-b9a0-cd0a7404a6b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:19.111722 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1072451-5e50-45f7-b9a0-cd0a7404a6b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:19.112028 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1072451-5e50-45f7-b9a0-cd0a7404a6b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:19.113399 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1072451-5e50-45f7-b9a0-cd0a7404a6b4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:19.125768 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 405/799] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:19.127042 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda3be9-1836-4311-bffc-9f69fac7910e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:19.127334 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda3be9-1836-4311-bffc-9f69fac7910e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:19.127615 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda3be9-1836-4311-bffc-9f69fac7910e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:19.138473 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:19.154336 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6fda3be9-1836-4311-bffc-9f69fac7910e None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:19.155562 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6fda3be9-1836-4311-bffc-9f69fac7910e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:19.184180 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2c2137-c6a3-4931-8bc2-2e62e3b776ee None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:19.208107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:19.225448 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a148a-79c9-4e88-96e4-a0dd60f17369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:19.226244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a148a-79c9-4e88-96e4-a0dd60f17369 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:19.226565 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a148a-79c9-4e88-96e4-a0dd60f17369 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:19.227003 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a148a-79c9-4e88-96e4-a0dd60f17369 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:19.227108 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 406/800] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:19.227690 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda3be9-1836-4311-bffc-9f69fac7910e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:19.245627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a148a-79c9-4e88-96e4-a0dd60f17369 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:19.245627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a148a-79c9-4e88-96e4-a0dd60f17369 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:19.266098 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-105a148a-79c9-4e88-96e4-a0dd60f17369 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:19.266098 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:19.266098 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:19.266098 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:19.266098 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:19.266098 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:19.266098 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:19.266098 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:19.266098 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:19.266098 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:19.266098 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:19.266098 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:19.266098 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:19.266098 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:19.266098 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:19.267530 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:19.267530 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:19.267530 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:19.267530 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:19.267530 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:19.267530 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:19.267530 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:19.267530 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:19.267530 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:19.267530 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:19.267530 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:19.267530 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:19.267530 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:19.267530 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:19.267530 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:19.267530 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:19.267530 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:19.268619 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:19.268619 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:19.268619 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:19.268619 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:19.268619 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 395/801] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:19.268619 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a974f-da07-405e-8fb7-82b007fc6284 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:19.269038 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a974f-da07-405e-8fb7-82b007fc6284 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:19.269038 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a974f-da07-405e-8fb7-82b007fc6284 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:19.270447 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 396/802] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:19.271848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a974f-da07-405e-8fb7-82b007fc6284 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:19.344031 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda3be9-1836-4311-bffc-9f69fac7910e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:19.383509 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42682ff2-6a95-45ed-945e-3b4405955342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:19.384369 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42682ff2-6a95-45ed-945e-3b4405955342 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:19.384790 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42682ff2-6a95-45ed-945e-3b4405955342 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:19.385144 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42682ff2-6a95-45ed-945e-3b4405955342 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:19.389436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a974f-da07-405e-8fb7-82b007fc6284 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:19.392338 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42682ff2-6a95-45ed-945e-3b4405955342 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:19.393557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42682ff2-6a95-45ed-945e-3b4405955342 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:19.403757 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:19.407703 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-42682ff2-6a95-45ed-945e-3b4405955342 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:19.407703 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:19.407703 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:19.407703 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:19.407703 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:19.407703 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:19.407703 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:19.407703 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:19.407703 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:19.407703 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:19.407703 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:19.407703 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:19.407703 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:21:19.407703 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:21:19.407703 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:21:19.408623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:21:19.408623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:19.408623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:19.408623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:19.408623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:19.408623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:19.408623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:19.408623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:19.408623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:19.408623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:19.408623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:19.408623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:19.408623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:19.408623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:19.408623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:19.408623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:19.408623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:19.409533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:21:19.409533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:21:19.409533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:21:19.409533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:21:19.409533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:21:19.409533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:21:19.409533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:19.409533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:19.409533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 407/803] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:21:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:19.410019 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42682ff2-6a95-45ed-945e-3b4405955342 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:19.433538 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c69d4d-c08e-4d1e-9cc7-811da9b48b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:19.434454 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c69d4d-c08e-4d1e-9cc7-811da9b48b2f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:19.434752 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c69d4d-c08e-4d1e-9cc7-811da9b48b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:19.435026 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c69d4d-c08e-4d1e-9cc7-811da9b48b2f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:19.435674 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c69d4d-c08e-4d1e-9cc7-811da9b48b2f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:19.437034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c69d4d-c08e-4d1e-9cc7-811da9b48b2f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:19.449305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-26c69d4d-c08e-4d1e-9cc7-811da9b48b2f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 12:21:19.449305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:19.449305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 12:21:19.449305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 12:21:19.449305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 12:21:19.449305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 12:21:19.449305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 12:21:19.449305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:19.449305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 12:21:19.449305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:19.449305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:21:19.449305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:19.449305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:19.449305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:19.449305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:19.449305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:19.450291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:19.450291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:19.450291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:19.450291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:19.450291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:19.450291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:19.450291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:19.450291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 12:21:19.450291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 12:21:19.450291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 12:21:19.450291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 12:21:19.450291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:19.450291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:19.450291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:19.450291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:19.450291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:19.451758 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:19.451758 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:19.451758 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:19.451758 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:19.451758 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:19.451758 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:19.451758 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:19.451758 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:19.451758 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:19.451758 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:19.451758 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:19.451758 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 12:21:19.451758 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:19.451758 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 29 12:21:19.451758 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:19.451758 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 397/804] 10.222.0.52 () {56 vars in 1129 bytes} [Wed Jun 29 12:21:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:19.452626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061ad57-4edd-444d-9439-f797692d9023 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:19.452626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061ad57-4edd-444d-9439-f797692d9023 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:19.452626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061ad57-4edd-444d-9439-f797692d9023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:19.465099 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:19.469327 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e061ad57-4edd-444d-9439-f797692d9023 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:19.470632 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e061ad57-4edd-444d-9439-f797692d9023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:19.523355 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42682ff2-6a95-45ed-945e-3b4405955342 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:19.550461 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 398/805] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:19.550461 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e061ad57-4edd-444d-9439-f797692d9023 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:19.558197 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ccb3f-1525-45ea-8e03-62398d2f4748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:19.559021 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ccb3f-1525-45ea-8e03-62398d2f4748 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:19.559433 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ccb3f-1525-45ea-8e03-62398d2f4748 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:19.559865 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8ccb3f-1525-45ea-8e03-62398d2f4748 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:19.560694 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ccb3f-1525-45ea-8e03-62398d2f4748 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:19.561110 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ccb3f-1525-45ea-8e03-62398d2f4748 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:19.562647 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8ccb3f-1525-45ea-8e03-62398d2f4748 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:19.582459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 408/806] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:19.582459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ccb3f-1525-45ea-8e03-62398d2f4748 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:19.672721 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e061ad57-4edd-444d-9439-f797692d9023 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:19.680125 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:19.691929 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8ccb3f-1525-45ea-8e03-62398d2f4748 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:19.695121 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:19.708477 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670e2bf-666e-422e-8e88-b8268b557908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:19.709143 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670e2bf-666e-422e-8e88-b8268b557908 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:19.709425 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670e2bf-666e-422e-8e88-b8268b557908 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:19.709703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c670e2bf-666e-422e-8e88-b8268b557908 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:19.710516 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c670e2bf-666e-422e-8e88-b8268b557908 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:19.710830 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c670e2bf-666e-422e-8e88-b8268b557908 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:19.712328 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c670e2bf-666e-422e-8e88-b8268b557908 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:19.724571 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 399/807] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:21:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:19.725607 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670e2bf-666e-422e-8e88-b8268b557908 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:19.736630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcaf800-62ed-4275-81b9-07b966edfabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:19.737291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcaf800-62ed-4275-81b9-07b966edfabb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:19.737571 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcaf800-62ed-4275-81b9-07b966edfabb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:19.737968 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcaf800-62ed-4275-81b9-07b966edfabb None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:19.738767 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcaf800-62ed-4275-81b9-07b966edfabb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:19.739072 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcaf800-62ed-4275-81b9-07b966edfabb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:19.740575 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcaf800-62ed-4275-81b9-07b966edfabb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:19.745909 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 409/808] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Jun 29 12:21:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:19.746894 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcaf800-62ed-4275-81b9-07b966edfabb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:19.841469 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c670e2bf-666e-422e-8e88-b8268b557908 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:19.850928 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcaf800-62ed-4275-81b9-07b966edfabb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:19.877518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364925f-fa85-45c1-b23b-742cd02f311c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:19.878325 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364925f-fa85-45c1-b23b-742cd02f311c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:19.878643 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364925f-fa85-45c1-b23b-742cd02f311c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:19.878926 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d364925f-fa85-45c1-b23b-742cd02f311c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:19.887291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283018ad-de48-4bb8-986c-6fef69bf6c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:19.888077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283018ad-de48-4bb8-986c-6fef69bf6c38 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:19.888360 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283018ad-de48-4bb8-986c-6fef69bf6c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:19.888635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283018ad-de48-4bb8-986c-6fef69bf6c38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:19.890143 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d364925f-fa85-45c1-b23b-742cd02f311c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:21:19.895433 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283018ad-de48-4bb8-986c-6fef69bf6c38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:19.897136 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283018ad-de48-4bb8-986c-6fef69bf6c38 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:19.898095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d364925f-fa85-45c1-b23b-742cd02f311c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:21:19.898095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:19.898095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:19.898095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:19.898095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:19.898095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:19.898095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:19.898095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:19.898095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:19.898095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:19.898095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:19.898095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:19.898095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:21:19.898095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:21:19.898095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:21:19.898095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:21:19.899496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:21:19.899496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:21:19.899496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:21:19.899496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:21:19.899496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:19.899496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:19.899496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:19.899496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:19.899496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:19.899496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:19.899496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:19.899496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:19.899496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:19.899496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:19.899496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:19.899496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:19.900900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:19.900900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:19.900900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:19.900900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:19.900900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:21:19.900900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:21:19.900900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:21:19.900900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:21:19.900900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:21:19.900900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:21:19.900900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:21:19.900900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:19.900900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 400/809] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:21:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:19.901828 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364925f-fa85-45c1-b23b-742cd02f311c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:19.918121 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-283018ad-de48-4bb8-986c-6fef69bf6c38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:19.918121 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:19.918121 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:19.918121 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:19.918121 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:19.918121 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:19.918121 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:19.918121 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:19.918121 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:19.918121 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:19.918121 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:19.918121 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:19.918121 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:21:19.918121 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:21:19.918121 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:21:19.919278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:21:19.919278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:19.919278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:19.919278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:19.919278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:19.919278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:19.919278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:19.919278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:19.919278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:19.919278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:19.919278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:19.919278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:19.919278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:19.919278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:19.919278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:19.919278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:19.919278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:19.920387 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:21:19.920387 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:21:19.920387 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:21:19.920387 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:21:19.920387 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:21:19.920387 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:21:19.920387 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:19.920387 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:19.920387 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 410/810] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:21:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:19.920387 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283018ad-de48-4bb8-986c-6fef69bf6c38 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:19.933495 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:19.949895 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:20.007546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d364925f-fa85-45c1-b23b-742cd02f311c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:20.023332 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283018ad-de48-4bb8-986c-6fef69bf6c38 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:20.041217 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f81db-a948-4a4b-86d0-7b1a1fa730a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:20.042103 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f81db-a948-4a4b-86d0-7b1a1fa730a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:20.042390 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f81db-a948-4a4b-86d0-7b1a1fa730a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:20.042699 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f81db-a948-4a4b-86d0-7b1a1fa730a4 None admin] PATH_INFO: `/v3/projects/f043ade9e4f748d4adc010ffb0a47a46/users/209b0df45f76404da880a9ae0c637d67/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:20.044664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f81db-a948-4a4b-86d0-7b1a1fa730a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f043ade9e4f748d4adc010ffb0a47a46, user_id=209b0df45f76404da880a9ae0c637d67, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:20.046096 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9f81db-a948-4a4b-86d0-7b1a1fa730a4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:20.056056 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46558676-4b74-4d88-aa7b-86f778f2110c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:20.057808 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46558676-4b74-4d88-aa7b-86f778f2110c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:20.058092 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46558676-4b74-4d88-aa7b-86f778f2110c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:20.058368 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46558676-4b74-4d88-aa7b-86f778f2110c None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:20.059101 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46558676-4b74-4d88-aa7b-86f778f2110c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:20.059266 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 401/811] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 12:21:19 2022] PUT /identity/v3/projects/f043ade9e4f748d4adc010ffb0a47a46/users/209b0df45f76404da880a9ae0c637d67/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:21:20.059418 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46558676-4b74-4d88-aa7b-86f778f2110c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:20.060054 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f81db-a948-4a4b-86d0-7b1a1fa730a4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:20.060778 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46558676-4b74-4d88-aa7b-86f778f2110c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:20.066979 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 411/812] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Jun 29 12:21:19 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:20.068581 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefa132-339e-4e2d-a979-350710882a3f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:20.068878 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefa132-339e-4e2d-a979-350710882a3f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:20.069159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefa132-339e-4e2d-a979-350710882a3f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:20.074595 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 412/813] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:20.075824 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feefa132-339e-4e2d-a979-350710882a3f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:20.173721 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:20.192255 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f81db-a948-4a4b-86d0-7b1a1fa730a4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:20.192671 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:20.203795 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feefa132-339e-4e2d-a979-350710882a3f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:20.236348 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c5e6c-d245-49c0-98c0-52ce902079e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:20.237329 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c5e6c-d245-49c0-98c0-52ce902079e4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:20.237809 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c5e6c-d245-49c0-98c0-52ce902079e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:20.238094 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c5e6c-d245-49c0-98c0-52ce902079e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:20.238880 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c5e6c-d245-49c0-98c0-52ce902079e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:20.239189 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c5e6c-d245-49c0-98c0-52ce902079e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:20.240588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c5e6c-d245-49c0-98c0-52ce902079e4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:20.242568 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4818ceff-3583-4cd3-97fc-9a8aff71fc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:20.243314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818ceff-3583-4cd3-97fc-9a8aff71fc88 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:20.243667 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818ceff-3583-4cd3-97fc-9a8aff71fc88 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:20.244454 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4818ceff-3583-4cd3-97fc-9a8aff71fc88 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:20.245241 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4818ceff-3583-4cd3-97fc-9a8aff71fc88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:20.245553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4818ceff-3583-4cd3-97fc-9a8aff71fc88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:20.246998 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4818ceff-3583-4cd3-97fc-9a8aff71fc88 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:20.249715 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 402/814] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:21:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:20.250511 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c5e6c-d245-49c0-98c0-52ce902079e4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:20.264941 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 413/815] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:21:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:20.266435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d6930-7009-42cb-a381-f307144c4725 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:20.266739 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d6930-7009-42cb-a381-f307144c4725 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:20.267022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3d6930-7009-42cb-a381-f307144c4725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:20.284167 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6f3d6930-7009-42cb-a381-f307144c4725 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:20.285341 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6f3d6930-7009-42cb-a381-f307144c4725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:20.357557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 414/816] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:20.358483 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d6930-7009-42cb-a381-f307144c4725 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:20.373738 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c5e6c-d245-49c0-98c0-52ce902079e4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:20.415058 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05634a-e85e-4c15-8ab2-58f2d981944e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:20.416275 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05634a-e85e-4c15-8ab2-58f2d981944e None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:20.416567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05634a-e85e-4c15-8ab2-58f2d981944e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:20.416848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb05634a-e85e-4c15-8ab2-58f2d981944e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:20.417601 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb05634a-e85e-4c15-8ab2-58f2d981944e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:20.419181 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb05634a-e85e-4c15-8ab2-58f2d981944e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:20.440265 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 403/817] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Jun 29 12:21:20 2022] POST /identity/v3/endpoints => generated 346 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:21:20.440265 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05634a-e85e-4c15-8ab2-58f2d981944e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:20.453030 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:20.472984 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:20.498862 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3d6930-7009-42cb-a381-f307144c4725 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:20.549218 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2c6bc-c6e2-4b44-a5fe-475d8dbe30f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:20.550057 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2c6bc-c6e2-4b44-a5fe-475d8dbe30f9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:20.550340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2c6bc-c6e2-4b44-a5fe-475d8dbe30f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:20.550662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2c6bc-c6e2-4b44-a5fe-475d8dbe30f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:20.562956 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2c6bc-c6e2-4b44-a5fe-475d8dbe30f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:20.564447 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc2c6bc-c6e2-4b44-a5fe-475d8dbe30f9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:20.574411 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb05634a-e85e-4c15-8ab2-58f2d981944e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:20.585451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5dc2c6bc-c6e2-4b44-a5fe-475d8dbe30f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:20.585451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:20.585451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:20.585451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:20.585451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:20.585451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:20.585451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:20.585451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:20.585451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:20.585451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:20.585451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:20.585451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:20.585451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:20.585451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:20.585451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:20.586451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:20.586451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:20.586451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:20.586451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:20.586451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:20.586451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:20.586451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:20.586451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:20.586451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:20.586451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:20.586451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:20.586451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:20.586451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:20.586451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:20.586451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:20.586451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:20.586451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:20.587537 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:20.587537 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:20.587537 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:20.587537 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:20.591812 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 415/818] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:20.592926 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2c6bc-c6e2-4b44-a5fe-475d8dbe30f9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:20.626492 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf388791-f75a-49f3-83ff-321698701c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:20.627486 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf388791-f75a-49f3-83ff-321698701c2b None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:20.627773 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf388791-f75a-49f3-83ff-321698701c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:20.628049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf388791-f75a-49f3-83ff-321698701c2b None admin] PATH_INFO: `/v3/projects/f043ade9e4f748d4adc010ffb0a47a46/users/8c10852a728844c6ae0a48e3772d95b7/roles/8883c151055d4ba493f514ce1bb62e78` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:20.644976 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf388791-f75a-49f3-83ff-321698701c2b None admin] RBAC: Authorizing `identity:create_grant(project_id=f043ade9e4f748d4adc010ffb0a47a46, user_id=8c10852a728844c6ae0a48e3772d95b7, role_id=8883c151055d4ba493f514ce1bb62e78)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:20.646730 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf388791-f75a-49f3-83ff-321698701c2b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:20.662711 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 404/819] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 12:21:20 2022] PUT /identity/v3/projects/f043ade9e4f748d4adc010ffb0a47a46/users/8c10852a728844c6ae0a48e3772d95b7/roles/8883c151055d4ba493f514ce1bb62e78 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:21:20.667044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf388791-f75a-49f3-83ff-321698701c2b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:20.714469 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2c6bc-c6e2-4b44-a5fe-475d8dbe30f9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:20.737230 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:20.753892 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:20.762281 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec877e8-ede7-48c1-a1a4-c88d95663797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:20.763396 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec877e8-ede7-48c1-a1a4-c88d95663797 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:20.763935 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec877e8-ede7-48c1-a1a4-c88d95663797 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:20.764386 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec877e8-ede7-48c1-a1a4-c88d95663797 None admin] PATH_INFO: `/v3/services/network` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:20.765203 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec877e8-ede7-48c1-a1a4-c88d95663797 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:20.766912 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec877e8-ede7-48c1-a1a4-c88d95663797 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:20.785238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6ec877e8-ede7-48c1-a1a4-c88d95663797 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 12:21:20.785238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:20.785238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 12:21:20.785238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 12:21:20.785238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 12:21:20.785238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 12:21:20.785238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 12:21:20.785238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:20.785238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 12:21:20.785238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:20.785238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:21:20.785238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:20.785238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:20.785238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:20.785238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:20.785238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:20.785238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:20.786444 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:20.786444 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:20.786444 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:20.786444 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:20.786444 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:20.786444 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:20.786444 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 12:21:20.786444 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 12:21:20.786444 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 12:21:20.786444 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 12:21:20.786444 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:20.786444 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:20.786444 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:20.786444 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:20.786444 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:20.786444 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:20.787437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:20.787437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:20.787437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:20.787437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:20.787437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:20.787437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:20.787437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:20.787437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:20.787437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:20.787437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:20.787437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 12:21:20.787437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:20.787437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 29 12:21:20.787437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:20.790182 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 416/820] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:21:20 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:20.793704 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf388791-f75a-49f3-83ff-321698701c2b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:20.799892 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec877e8-ede7-48c1-a1a4-c88d95663797 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:20.854493 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ce80e-2f62-4ceb-8b04-36b60fecf9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:20.855174 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ce80e-2f62-4ceb-8b04-36b60fecf9a7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:20.855463 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ce80e-2f62-4ceb-8b04-36b60fecf9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:20.855744 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ce80e-2f62-4ceb-8b04-36b60fecf9a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:20.856656 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ce80e-2f62-4ceb-8b04-36b60fecf9a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:20.856765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ce80e-2f62-4ceb-8b04-36b60fecf9a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:20.858156 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ce80e-2f62-4ceb-8b04-36b60fecf9a7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:20.870974 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 405/821] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:20.878498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ce80e-2f62-4ceb-8b04-36b60fecf9a7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:20.938252 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec877e8-ede7-48c1-a1a4-c88d95663797 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:20.973827 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc74703-4c26-416c-8d0d-2c938bc9f582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:20.974520 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc74703-4c26-416c-8d0d-2c938bc9f582 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:20.974881 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc74703-4c26-416c-8d0d-2c938bc9f582 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:20.975425 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc74703-4c26-416c-8d0d-2c938bc9f582 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:20.976012 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc74703-4c26-416c-8d0d-2c938bc9f582 None admin] RBAC: Adding query filter params (name=network) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:20.976391 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc74703-4c26-416c-8d0d-2c938bc9f582 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:20.977843 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc74703-4c26-416c-8d0d-2c938bc9f582 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:20.983428 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 417/822] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jun 29 12:21:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:20.993639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc74703-4c26-416c-8d0d-2c938bc9f582 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:21.004411 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ce80e-2f62-4ceb-8b04-36b60fecf9a7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:21.038205 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:21.058239 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:21.105787 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc74703-4c26-416c-8d0d-2c938bc9f582 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:21.119623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead164e-e422-4e62-be70-15cef82813a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:21.120441 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead164e-e422-4e62-be70-15cef82813a3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:21.120728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead164e-e422-4e62-be70-15cef82813a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:21.121002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dead164e-e422-4e62-be70-15cef82813a3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:21.133763 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead164e-e422-4e62-be70-15cef82813a3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:21.135286 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dead164e-e422-4e62-be70-15cef82813a3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:21.151687 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-dead164e-e422-4e62-be70-15cef82813a3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 12:21:21.151687 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:21.151687 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:21.151687 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:21.151687 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:21.151687 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:21.151687 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:21.151687 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:21.151687 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:21.151687 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:21.151687 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:21.151687 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:21.151687 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:21:21.151687 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:21:21.151687 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:21:21.151687 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:21:21.152665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:21.152665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:21.152665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:21:21.152665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:21.152665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:21:21.152665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:21.152665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:21.152665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:21.152665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:21.152665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:21.152665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:21.152665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:21.152665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:21.152665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:21.152665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:21.152665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:21.152665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:21.152665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:21.153603 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:21.153603 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:21.153603 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:21:21.153603 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:21:21.153603 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:21:21.153603 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:21:21.153603 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:21:21.153603 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:21:21.153603 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 12:21:21.153603 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:21.153603 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 406/823] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Jun 29 12:21:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:21.160229 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead164e-e422-4e62-be70-15cef82813a3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:21.184686 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f059e12-bb10-4792-88af-307c88f93919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:21.185359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f059e12-bb10-4792-88af-307c88f93919 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:21.185643 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f059e12-bb10-4792-88af-307c88f93919 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:21.185983 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f059e12-bb10-4792-88af-307c88f93919 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:21.186733 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f059e12-bb10-4792-88af-307c88f93919 None admin] RBAC: Adding query filter params (type=network) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:21.188673 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f059e12-bb10-4792-88af-307c88f93919 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:21.190082 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f059e12-bb10-4792-88af-307c88f93919 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:21.205256 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 418/824] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jun 29 12:21:20 2022] GET /identity/v3/services?type=network => generated 350 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:21.207765 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f059e12-bb10-4792-88af-307c88f93919 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:21.375084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f059e12-bb10-4792-88af-307c88f93919 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:21.403088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:21.416916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23809878-dc1b-4eac-94a1-01617df325ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:21.417578 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23809878-dc1b-4eac-94a1-01617df325ce None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:21.417760 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dead164e-e422-4e62-be70-15cef82813a3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:21.418198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23809878-dc1b-4eac-94a1-01617df325ce None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:21.418198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23809878-dc1b-4eac-94a1-01617df325ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:21.418891 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23809878-dc1b-4eac-94a1-01617df325ce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:21.420403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23809878-dc1b-4eac-94a1-01617df325ce None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:21.453373 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 419/825] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Jun 29 12:21:21 2022] POST /identity/v3/endpoints => generated 327 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:21:21.497370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1957d-3910-4a52-afe3-684d6d17866a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:21.498140 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1957d-3910-4a52-afe3-684d6d17866a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:21.498449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1957d-3910-4a52-afe3-684d6d17866a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:21.498781 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1957d-3910-4a52-afe3-684d6d17866a None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:21.499675 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d1957d-3910-4a52-afe3-684d6d17866a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:21.500045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d1957d-3910-4a52-afe3-684d6d17866a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:21.501537 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d1957d-3910-4a52-afe3-684d6d17866a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:21.528321 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 407/826] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Jun 29 12:21:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:21.533661 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23809878-dc1b-4eac-94a1-01617df325ce None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:21.684415 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23809878-dc1b-4eac-94a1-01617df325ce None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:21.720538 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf2c59-22f8-4fab-8821-b3117a097166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:21.721600 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf2c59-22f8-4fab-8821-b3117a097166 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:21.722129 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf2c59-22f8-4fab-8821-b3117a097166 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:21.722655 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf2c59-22f8-4fab-8821-b3117a097166 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:21.772655 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:21.799872 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf2c59-22f8-4fab-8821-b3117a097166 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:21.801338 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf2c59-22f8-4fab-8821-b3117a097166 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:22.031597 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 420/827] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:21:21 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 12:21:22.037641 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1957d-3910-4a52-afe3-684d6d17866a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:22.078894 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:22.154224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1957d-3910-4a52-afe3-684d6d17866a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:22.194311 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf5360-5fef-4540-803f-656139cc8718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:22.195154 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf5360-5fef-4540-803f-656139cc8718 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:22.195442 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf5360-5fef-4540-803f-656139cc8718 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:22.195719 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf5360-5fef-4540-803f-656139cc8718 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:22.206285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebf5360-5fef-4540-803f-656139cc8718 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:22.207782 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebf5360-5fef-4540-803f-656139cc8718 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:22.227034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4ebf5360-5fef-4540-803f-656139cc8718 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:22.227034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:22.227034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:22.227034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:22.227034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:22.227034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:22.227034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:22.227034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:22.227034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:22.227034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:22.227034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:22.227034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:22.227034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:22.227034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:22.227034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:22.228012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:22.228012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:22.228012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:22.228012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:22.228012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:22.228012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:22.228012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:22.228012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:22.228012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:22.228012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:22.228012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:22.228012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:22.228012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:22.228012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:22.228012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:22.228012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:22.228012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:22.229267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:22.229267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:22.229267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:22.229267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:22.229267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 408/828] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:22.235516 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf2c59-22f8-4fab-8821-b3117a097166 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:22.314730 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:22.336603 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf2c59-22f8-4fab-8821-b3117a097166 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:22.372072 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055dcca-3c92-418b-a1b5-1d9131de07c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:22.372740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055dcca-3c92-418b-a1b5-1d9131de07c3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:22.373020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055dcca-3c92-418b-a1b5-1d9131de07c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:22.373295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055dcca-3c92-418b-a1b5-1d9131de07c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:22.374014 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3055dcca-3c92-418b-a1b5-1d9131de07c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:22.374318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3055dcca-3c92-418b-a1b5-1d9131de07c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:22.375714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3055dcca-3c92-418b-a1b5-1d9131de07c3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:22.387919 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 421/829] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:22.393255 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf5360-5fef-4540-803f-656139cc8718 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:22.463697 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423f816-f7e4-459e-9e5a-797785e030ad None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:22.464029 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423f816-f7e4-459e-9e5a-797785e030ad None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:22.464262 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423f816-f7e4-459e-9e5a-797785e030ad None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:22.465902 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 422/830] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:22.471112 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cccd88-aac2-4f76-aa2b-058f7e195bbb None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:22.471426 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cccd88-aac2-4f76-aa2b-058f7e195bbb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:22.471669 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cccd88-aac2-4f76-aa2b-058f7e195bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:22.485276 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:22.490602 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e5cccd88-aac2-4f76-aa2b-058f7e195bbb None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:22.491715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e5cccd88-aac2-4f76-aa2b-058f7e195bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:22.508445 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf5360-5fef-4540-803f-656139cc8718 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:22.545867 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53301798-a3e8-4b41-ab03-1493d8430a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:22.547115 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53301798-a3e8-4b41-ab03-1493d8430a1c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:22.547682 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53301798-a3e8-4b41-ab03-1493d8430a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:22.548227 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53301798-a3e8-4b41-ab03-1493d8430a1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:22.556070 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53301798-a3e8-4b41-ab03-1493d8430a1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:22.558600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53301798-a3e8-4b41-ab03-1493d8430a1c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:22.574379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-53301798-a3e8-4b41-ab03-1493d8430a1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:22.574379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:22.574379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:22.574379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:22.574379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:22.574379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:22.574379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:22.574379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:22.574379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:22.574379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:22.574379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:22.574379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:22.574379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:21:22.574379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:21:22.574379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:21:22.575487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:21:22.575487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:22.575487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:22.575487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:22.575487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:22.575487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:22.575487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:22.575487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:22.575487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:22.575487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:22.575487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:22.575487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:22.575487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:22.575487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:22.575487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:22.575487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:22.575487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:22.576408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:21:22.576408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:21:22.576408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:21:22.576408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:21:22.576408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:21:22.576408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:21:22.576408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:22.576408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:22.576879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 423/831] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:22 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 12:21:22.578953 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 409/832] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:21:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:22.584229 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cccd88-aac2-4f76-aa2b-058f7e195bbb None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:22.593296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53301798-a3e8-4b41-ab03-1493d8430a1c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:22.654487 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:22.690807 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cccd88-aac2-4f76-aa2b-058f7e195bbb None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:22.726805 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585e794-f0b6-4aac-b538-b8a415c9f1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:22.727777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585e794-f0b6-4aac-b538-b8a415c9f1ff None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:22.728669 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585e794-f0b6-4aac-b538-b8a415c9f1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:22.728951 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3585e794-f0b6-4aac-b538-b8a415c9f1ff None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:22.730331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3585e794-f0b6-4aac-b538-b8a415c9f1ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:22.730680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3585e794-f0b6-4aac-b538-b8a415c9f1ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:22.732598 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3585e794-f0b6-4aac-b538-b8a415c9f1ff None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:22.735534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53301798-a3e8-4b41-ab03-1493d8430a1c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:22.749696 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 424/833] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:21:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:22.756427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585e794-f0b6-4aac-b538-b8a415c9f1ff None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:22.780520 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67e7fb-fbb6-4ecd-97f4-bb8a7b59cc7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:22.781761 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67e7fb-fbb6-4ecd-97f4-bb8a7b59cc7d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:22.782273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67e7fb-fbb6-4ecd-97f4-bb8a7b59cc7d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:22.782794 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67e7fb-fbb6-4ecd-97f4-bb8a7b59cc7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:22.791863 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:22.795789 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a67e7fb-fbb6-4ecd-97f4-bb8a7b59cc7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:21:22.803765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4a67e7fb-fbb6-4ecd-97f4-bb8a7b59cc7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:21:22.803765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:22.803765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:22.803765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:22.803765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:22.803765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:22.803765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:22.803765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:22.803765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:22.803765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:22.803765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:22.803765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:22.803765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:21:22.803765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:21:22.803765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:21:22.803765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:21:22.804693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:21:22.804693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:21:22.804693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:21:22.804693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:21:22.804693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:22.804693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:22.804693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:22.804693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:22.804693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:22.804693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:22.804693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:22.804693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:22.804693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:22.804693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:22.804693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:22.804693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:22.805525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:22.805525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:22.805525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:22.805525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:22.805525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:21:22.805525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:21:22.805525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:21:22.805525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:21:22.805525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:21:22.805525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:21:22.805525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:21:22.805525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:22.812773 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 410/834] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:21:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:22.812773 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24f92d-748b-41ce-8c1b-a30a63af39e8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:22.812773 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24f92d-748b-41ce-8c1b-a30a63af39e8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:22.812773 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba24f92d-748b-41ce-8c1b-a30a63af39e8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:22.814571 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 411/835] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:22.815508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba24f92d-748b-41ce-8c1b-a30a63af39e8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:22.878962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3585e794-f0b6-4aac-b538-b8a415c9f1ff None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:22.908862 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:22.918363 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be81cd-2469-4e7a-9c88-d2e1952d2744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:22.919432 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be81cd-2469-4e7a-9c88-d2e1952d2744 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:22.919738 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be81cd-2469-4e7a-9c88-d2e1952d2744 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:22.920019 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be81cd-2469-4e7a-9c88-d2e1952d2744 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:22.920843 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be81cd-2469-4e7a-9c88-d2e1952d2744 None admin] RBAC: Adding query filter params (role.id=3a7f4bf556344407a6d9540c89d13641, scope.project.id=f043ade9e4f748d4adc010ffb0a47a46, user.id=3ac7391a3c41499d94e5f97478797313) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:22.921155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be81cd-2469-4e7a-9c88-d2e1952d2744 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:22.922619 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be81cd-2469-4e7a-9c88-d2e1952d2744 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:22.932480 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba24f92d-748b-41ce-8c1b-a30a63af39e8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:22.934799 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 425/836] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:21:22 2022] GET /identity/v3/role_assignments?user.id=3ac7391a3c41499d94e5f97478797313&scope.project.id=f043ade9e4f748d4adc010ffb0a47a46&role.id=3a7f4bf556344407a6d9540c89d13641 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:22.937540 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bee93-8159-4b5b-8164-1f542a2ef912 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:22.938100 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bee93-8159-4b5b-8164-1f542a2ef912 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:22.938632 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bee93-8159-4b5b-8164-1f542a2ef912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:22.963320 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-fb0bee93-8159-4b5b-8164-1f542a2ef912 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:22.964632 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fb0bee93-8159-4b5b-8164-1f542a2ef912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:22.978593 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4e0c8-1eeb-4eac-9e80-cbc09abf1133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:22.979522 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4e0c8-1eeb-4eac-9e80-cbc09abf1133 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:22.979911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4e0c8-1eeb-4eac-9e80-cbc09abf1133 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:22.980291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4e0c8-1eeb-4eac-9e80-cbc09abf1133 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:22.981393 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e4e0c8-1eeb-4eac-9e80-cbc09abf1133 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:22.981815 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e4e0c8-1eeb-4eac-9e80-cbc09abf1133 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:22.983597 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e4e0c8-1eeb-4eac-9e80-cbc09abf1133 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:22.998155 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 412/837] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:21:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:23.004185 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4e0c8-1eeb-4eac-9e80-cbc09abf1133 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:23.070494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 426/838] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:22 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:23.092059 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0bee93-8159-4b5b-8164-1f542a2ef912 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:23.093573 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:23.141310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4e0c8-1eeb-4eac-9e80-cbc09abf1133 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:23.179956 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b38e3-9627-4978-a1c2-4b68af2a8bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:23.180754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b38e3-9627-4978-a1c2-4b68af2a8bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:23.181465 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b38e3-9627-4978-a1c2-4b68af2a8bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:23.181465 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b38e3-9627-4978-a1c2-4b68af2a8bc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:23.192008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b38e3-9627-4978-a1c2-4b68af2a8bc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:23.193790 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b38e3-9627-4978-a1c2-4b68af2a8bc8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:23.202703 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0bee93-8159-4b5b-8164-1f542a2ef912 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:23.212177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a91b38e3-9627-4978-a1c2-4b68af2a8bc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:23.212177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:23.212177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:23.212177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:23.212177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:23.212177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:23.212177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:23.212177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:23.212177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:23.212177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:23.212177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:23.212177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:23.212177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:23.212177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:23.212177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:23.213270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:23.213270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:23.213270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:23.213270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:23.213270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:23.213270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:23.213270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:23.213270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:23.213270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:23.213270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:23.213270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:23.213270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:23.213270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:23.213270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:23.213270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:23.213270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:23.213270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:23.214302 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:23.214302 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:23.214302 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:23.214302 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:23.214302 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 413/839] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:23.215046 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356f3b6-12b2-42d9-8dc6-1f86afab6c79 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:23.215334 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356f3b6-12b2-42d9-8dc6-1f86afab6c79 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:23.215609 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5356f3b6-12b2-42d9-8dc6-1f86afab6c79 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:23.220614 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 414/840] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:23.220614 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5356f3b6-12b2-42d9-8dc6-1f86afab6c79 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:23.233848 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:23.247894 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ceee8-6f3e-404f-bea9-c375ad39feae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:23.248960 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ceee8-6f3e-404f-bea9-c375ad39feae None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:23.249243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ceee8-6f3e-404f-bea9-c375ad39feae None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:23.249518 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ceee8-6f3e-404f-bea9-c375ad39feae None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:23.250167 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ceee8-6f3e-404f-bea9-c375ad39feae None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:23.251573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ceee8-6f3e-404f-bea9-c375ad39feae None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:23.264125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-772ceee8-6f3e-404f-bea9-c375ad39feae None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 12:21:23.264125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:23.264125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 12:21:23.264125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 12:21:23.264125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 12:21:23.264125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 12:21:23.264125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 12:21:23.264125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:23.264125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 12:21:23.264125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:23.264125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:21:23.264125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:23.264125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:23.264125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:23.264125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:23.264125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:23.264125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:23.265140 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:23.265140 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:23.265140 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:23.265140 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:23.265140 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:23.265140 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:23.265140 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 12:21:23.265140 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 12:21:23.265140 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 12:21:23.265140 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 12:21:23.265140 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:23.265140 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:23.265140 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:23.265140 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:23.265140 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:23.265140 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:23.265963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:23.265963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:23.265963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:23.265963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:23.265963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:23.265963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:23.265963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:23.265963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:23.265963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:23.265963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:23.265963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 12:21:23.265963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:23.265963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 12:21:23.265963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:23.265963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 427/841] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 12:21:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:23.267367 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13821f68-3108-41d2-a7fb-6443e54f5ace None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:23.267367 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13821f68-3108-41d2-a7fb-6443e54f5ace None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:23.267519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13821f68-3108-41d2-a7fb-6443e54f5ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:23.285102 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-13821f68-3108-41d2-a7fb-6443e54f5ace None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:23.286272 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-13821f68-3108-41d2-a7fb-6443e54f5ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:23.333785 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5356f3b6-12b2-42d9-8dc6-1f86afab6c79 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:23.357174 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 428/842] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:23 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:23.358459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13821f68-3108-41d2-a7fb-6443e54f5ace None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:23.466922 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:23.474443 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13821f68-3108-41d2-a7fb-6443e54f5ace None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:23.489075 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e749b-9839-4a7b-8af8-abe0679a8626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:23.490358 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e749b-9839-4a7b-8af8-abe0679a8626 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:23.490876 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e749b-9839-4a7b-8af8-abe0679a8626 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:23.491159 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e749b-9839-4a7b-8af8-abe0679a8626 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:23.491890 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e749b-9839-4a7b-8af8-abe0679a8626 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:23.492043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:23.492242 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e749b-9839-4a7b-8af8-abe0679a8626 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:23.493611 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783e749b-9839-4a7b-8af8-abe0679a8626 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:23.506851 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 415/843] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:23.507680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e749b-9839-4a7b-8af8-abe0679a8626 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:23.516214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0dac5a-c3ac-4eb0-b85e-c398e635e746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:23.517132 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0dac5a-c3ac-4eb0-b85e-c398e635e746 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:23.517652 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0dac5a-c3ac-4eb0-b85e-c398e635e746 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:23.518165 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0dac5a-c3ac-4eb0-b85e-c398e635e746 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:23.519151 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0dac5a-c3ac-4eb0-b85e-c398e635e746 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:23.519641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0dac5a-c3ac-4eb0-b85e-c398e635e746 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:23.521304 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0dac5a-c3ac-4eb0-b85e-c398e635e746 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:23.527522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 429/844] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 12:21:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:23.528591 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0dac5a-c3ac-4eb0-b85e-c398e635e746 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:23.623154 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783e749b-9839-4a7b-8af8-abe0679a8626 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:23.638972 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0dac5a-c3ac-4eb0-b85e-c398e635e746 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:23.659469 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611f428-cdd7-483e-b61c-3ad393691243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:23.660271 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611f428-cdd7-483e-b61c-3ad393691243 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:23.660558 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611f428-cdd7-483e-b61c-3ad393691243 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:23.660841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611f428-cdd7-483e-b61c-3ad393691243 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:23.671852 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e611f428-cdd7-483e-b61c-3ad393691243 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:21:23.674473 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7dd09-dff2-4dc5-a3df-20f218379bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:23.675260 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7dd09-dff2-4dc5-a3df-20f218379bcb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:23.675544 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7dd09-dff2-4dc5-a3df-20f218379bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:23.675819 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7dd09-dff2-4dc5-a3df-20f218379bcb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:23.679536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e611f428-cdd7-483e-b61c-3ad393691243 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:21:23.679536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:23.679536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:23.679536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:23.679536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:23.679536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:23.679536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:23.679536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:23.679536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:23.679536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:23.679536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:23.679536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:23.679536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:21:23.679536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:21:23.679536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:21:23.679536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:21:23.680479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:21:23.680479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:21:23.680479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:21:23.680479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:21:23.680479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:23.680479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:23.680479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:23.680479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:23.680479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:23.680479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:23.680479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:23.680479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:23.680479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:23.680479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:23.680479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:23.680479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:23.687257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:23.687257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:23.687257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:23.687257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:23.687257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:21:23.687257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:21:23.687257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:21:23.687257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:21:23.687257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:21:23.687257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:21:23.687257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:21:23.687257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:23.687257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 416/845] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 12:21:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:23.687257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611f428-cdd7-483e-b61c-3ad393691243 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:23.688031 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7dd09-dff2-4dc5-a3df-20f218379bcb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:23.688031 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7dd09-dff2-4dc5-a3df-20f218379bcb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:23.703156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-52c7dd09-dff2-4dc5-a3df-20f218379bcb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 29 12:21:23.703156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:23.703156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:23.703156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:23.703156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:23.703156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:23.703156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:23.703156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:23.703156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:23.703156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:23.703156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:23.703156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:23.703156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:21:23.703156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:21:23.703156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:21:23.703156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:21:23.704155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:23.704155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:23.704155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:21:23.704155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:23.704155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:21:23.704155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:23.704155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:23.704155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:23.704155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:23.704155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:23.704155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:23.704155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:23.704155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:23.704155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:23.704155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:23.704155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:23.704155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:23.704155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:23.705739 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:23.705739 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:23.705739 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:21:23.705739 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:21:23.705739 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:21:23.705739 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:21:23.705739 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:21:23.705739 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:21:23.705739 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 29 12:21:23.705739 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:23.705739 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 430/846] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:21:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:23.705739 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7dd09-dff2-4dc5-a3df-20f218379bcb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:23.729603 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:23.753955 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:23.796041 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611f428-cdd7-483e-b61c-3ad393691243 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:23.818519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7dd09-dff2-4dc5-a3df-20f218379bcb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:23.830496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6a1cd-d301-470b-add0-5d9c48e0ab0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:23.831173 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6a1cd-d301-470b-add0-5d9c48e0ab0c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:23.831463 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6a1cd-d301-470b-add0-5d9c48e0ab0c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:23.831743 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6a1cd-d301-470b-add0-5d9c48e0ab0c None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:23.832459 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6a1cd-d301-470b-add0-5d9c48e0ab0c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:23.832767 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6a1cd-d301-470b-add0-5d9c48e0ab0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:23.834140 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b6a1cd-d301-470b-add0-5d9c48e0ab0c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:23.839001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 417/847] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 12:21:23 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:23.839793 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6a1cd-d301-470b-add0-5d9c48e0ab0c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:23.867804 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabbdb6-4b3b-4736-9466-222b7df39f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:23.869163 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabbdb6-4b3b-4736-9466-222b7df39f06 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:23.869682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabbdb6-4b3b-4736-9466-222b7df39f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:23.870152 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabbdb6-4b3b-4736-9466-222b7df39f06 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:23.871049 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cabbdb6-4b3b-4736-9466-222b7df39f06 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:23.871426 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cabbdb6-4b3b-4736-9466-222b7df39f06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:23.873164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cabbdb6-4b3b-4736-9466-222b7df39f06 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:23.888205 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 431/848] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 12:21:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:23.889819 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabbdb6-4b3b-4736-9466-222b7df39f06 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:23.961322 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:23.967580 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6a1cd-d301-470b-add0-5d9c48e0ab0c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:24.015660 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fccfab-b336-4a07-bb3b-6a2c52fb4547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:24.016347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fccfab-b336-4a07-bb3b-6a2c52fb4547 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:24.016631 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fccfab-b336-4a07-bb3b-6a2c52fb4547 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:24.016902 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fccfab-b336-4a07-bb3b-6a2c52fb4547 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:24.017749 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fccfab-b336-4a07-bb3b-6a2c52fb4547 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:24.018055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fccfab-b336-4a07-bb3b-6a2c52fb4547 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:24.019559 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fccfab-b336-4a07-bb3b-6a2c52fb4547 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:24.026754 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabbdb6-4b3b-4736-9466-222b7df39f06 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:24.029135 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:24.041814 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 418/849] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Jun 29 12:21:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:24.047179 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fccfab-b336-4a07-bb3b-6a2c52fb4547 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:24.068910 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb7f59-4319-4741-a591-d145180dc49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:24.069710 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb7f59-4319-4741-a591-d145180dc49f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:24.069995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb7f59-4319-4741-a591-d145180dc49f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:24.070271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb7f59-4319-4741-a591-d145180dc49f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:24.082233 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb7f59-4319-4741-a591-d145180dc49f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:24.083722 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb7f59-4319-4741-a591-d145180dc49f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:24.107331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-dafb7f59-4319-4741-a591-d145180dc49f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:24.107331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:24.107331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:24.107331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:24.107331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:24.107331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:24.107331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:24.107331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:24.107331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:24.107331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:24.107331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:24.107331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:24.107331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:24.107331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:24.107331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:24.108301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:24.108301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:24.108301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:24.108301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:24.108301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:24.108301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:24.108301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:24.108301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:24.108301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:24.108301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:24.108301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:24.108301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:24.108301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:24.108301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:24.108301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:24.108301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:24.108301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:24.109205 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:24.109205 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:24.109205 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:24.109205 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:24.112136 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 432/850] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:24.118693 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb7f59-4319-4741-a591-d145180dc49f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:24.176332 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fccfab-b336-4a07-bb3b-6a2c52fb4547 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:24.225834 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88939152-890a-4d4a-8aad-e7fe66842938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:24.227316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88939152-890a-4d4a-8aad-e7fe66842938 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:24.227755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88939152-890a-4d4a-8aad-e7fe66842938 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:24.229535 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88939152-890a-4d4a-8aad-e7fe66842938 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:24.240875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb7f59-4319-4741-a591-d145180dc49f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:24.264885 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:24.280372 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d1d4d-0548-4477-902f-5e1c58248a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:24.280613 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d1d4d-0548-4477-902f-5e1c58248a80 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:24.281005 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d1d4d-0548-4477-902f-5e1c58248a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:24.281379 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d1d4d-0548-4477-902f-5e1c58248a80 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:24.282254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d1d4d-0548-4477-902f-5e1c58248a80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:24.282706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d1d4d-0548-4477-902f-5e1c58248a80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:24.284183 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54d1d4d-0548-4477-902f-5e1c58248a80 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:24.297015 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 433/851] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:24.307944 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d1d4d-0548-4477-902f-5e1c58248a80 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:24.323951 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88939152-890a-4d4a-8aad-e7fe66842938 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:24.325575 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88939152-890a-4d4a-8aad-e7fe66842938 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:24.329244 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:24.444346 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 419/852] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:21:24 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 29 12:21:24.450631 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d1d4d-0548-4477-902f-5e1c58248a80 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:24.450800 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88939152-890a-4d4a-8aad-e7fe66842938 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:24.486252 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a6dbd-d75c-4bf9-a0d3-8794246517b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:24.487208 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a6dbd-d75c-4bf9-a0d3-8794246517b6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:24.487592 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a6dbd-d75c-4bf9-a0d3-8794246517b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:24.487948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a6dbd-d75c-4bf9-a0d3-8794246517b6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:24.496012 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a6dbd-d75c-4bf9-a0d3-8794246517b6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:24.497560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a6dbd-d75c-4bf9-a0d3-8794246517b6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:24.513059 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e62a6dbd-d75c-4bf9-a0d3-8794246517b6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 12:21:24.513059 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:24.513059 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:24.513059 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:24.513059 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:24.513059 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:24.513059 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:24.513059 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:24.513059 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:24.513059 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:24.513059 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:24.513059 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:24.513059 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:21:24.513059 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:21:24.513059 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:21:24.513059 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:21:24.514039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:24.514039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:24.514039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:21:24.514039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:24.514039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:21:24.514039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:24.514039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:24.514039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:24.514039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:24.514039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:24.514039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:24.514039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:24.514039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:24.514039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:24.514039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:24.514039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:24.514039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:24.514039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:24.515048 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:24.515048 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:24.515048 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:21:24.515048 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:21:24.515048 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:21:24.515048 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:21:24.515048 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:21:24.515048 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:21:24.515048 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 12:21:24.515048 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:24.515048 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 434/853] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jun 29 12:21:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:24.519812 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a6dbd-d75c-4bf9-a0d3-8794246517b6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:24.520032 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:24.586630 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88939152-890a-4d4a-8aad-e7fe66842938 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:24.615921 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:24.635706 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e00ec-e840-432e-8987-1a58854e65ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:24.637623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e00ec-e840-432e-8987-1a58854e65ea None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:24.638111 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e00ec-e840-432e-8987-1a58854e65ea None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:24.638836 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e00ec-e840-432e-8987-1a58854e65ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:24.645490 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a6dbd-d75c-4bf9-a0d3-8794246517b6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:24.653596 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e00ec-e840-432e-8987-1a58854e65ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:24.655843 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31e00ec-e840-432e-8987-1a58854e65ea None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:24.682337 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d31e00ec-e840-432e-8987-1a58854e65ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:24.682337 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:24.682337 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:24.682337 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:24.682337 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:24.682337 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:24.682337 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:24.682337 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:24.682337 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:24.682337 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:24.682337 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:24.682337 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:24.682337 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:24.682337 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:24.682337 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:24.683342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:24.683342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:24.683342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:24.683342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:24.683342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:24.683342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:24.683342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:24.683342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:24.683342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:24.683342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:24.683342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:24.683342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:24.683342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:24.683342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:24.683342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:24.683342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:24.683342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:24.684273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:24.684273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:24.684273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:24.684273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:24.685639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecb343-061c-4046-a0bd-8d9fab6e90e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:24.690435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecb343-061c-4046-a0bd-8d9fab6e90e9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:24.690435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecb343-061c-4046-a0bd-8d9fab6e90e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:24.690435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecb343-061c-4046-a0bd-8d9fab6e90e9 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:24.690435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ecb343-061c-4046-a0bd-8d9fab6e90e9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:24.690435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ecb343-061c-4046-a0bd-8d9fab6e90e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:24.690435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ecb343-061c-4046-a0bd-8d9fab6e90e9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:24.690964 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 420/854] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:24.691580 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e00ec-e840-432e-8987-1a58854e65ea None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:24.713871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 435/855] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Jun 29 12:21:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:24.715293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecb343-061c-4046-a0bd-8d9fab6e90e9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:24.813517 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31e00ec-e840-432e-8987-1a58854e65ea None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:24.823804 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:24.851108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8856f-ea8f-47b4-bc24-c89490dc4643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:24.851843 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8856f-ea8f-47b4-bc24-c89490dc4643 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:24.852247 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8856f-ea8f-47b4-bc24-c89490dc4643 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:24.852626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8856f-ea8f-47b4-bc24-c89490dc4643 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:24.853906 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecb343-061c-4046-a0bd-8d9fab6e90e9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:24.854119 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d8856f-ea8f-47b4-bc24-c89490dc4643 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:24.854552 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d8856f-ea8f-47b4-bc24-c89490dc4643 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:24.856113 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d8856f-ea8f-47b4-bc24-c89490dc4643 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:24.873196 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 421/856] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:24.874247 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8856f-ea8f-47b4-bc24-c89490dc4643 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:24.895782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:24.903076 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0582d99e-61df-48c9-bd83-21ec3724b4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:24.904162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582d99e-61df-48c9-bd83-21ec3724b4b1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:24.904798 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582d99e-61df-48c9-bd83-21ec3724b4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:24.905212 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582d99e-61df-48c9-bd83-21ec3724b4b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:24.979411 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0582d99e-61df-48c9-bd83-21ec3724b4b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:24.981046 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0582d99e-61df-48c9-bd83-21ec3724b4b1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:25.025807 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8856f-ea8f-47b4-bc24-c89490dc4643 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:25.067946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023d2948-899d-446e-b4e2-29e0b45a4f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:25.069037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d2948-899d-446e-b4e2-29e0b45a4f42 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:25.069323 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d2948-899d-446e-b4e2-29e0b45a4f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:25.069600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d2948-899d-446e-b4e2-29e0b45a4f42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:25.077640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d2948-899d-446e-b4e2-29e0b45a4f42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:25.079122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023d2948-899d-446e-b4e2-29e0b45a4f42 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:25.093948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-023d2948-899d-446e-b4e2-29e0b45a4f42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:25.093948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:25.093948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:25.093948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:25.093948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:25.093948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:25.093948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:25.093948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:25.093948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:25.093948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:25.093948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:25.093948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:25.093948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:21:25.093948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:21:25.093948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:21:25.095097 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:21:25.095097 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:25.095097 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:25.095097 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:25.095097 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:25.095097 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:25.095097 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:25.095097 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:25.095097 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:25.095097 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:25.095097 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:25.095097 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:25.095097 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:25.095097 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:25.095097 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:25.095097 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:25.095097 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:25.095993 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:21:25.095993 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:21:25.095993 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:21:25.095993 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:21:25.095993 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:21:25.095993 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:21:25.095993 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:25.095993 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:25.099915 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 422/857] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:21:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:25.100457 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 436/858] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:21:24 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 29 12:21:25.101175 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90147c33-d292-454a-84b5-380401b8e7b7 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:25.101275 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0582d99e-61df-48c9-bd83-21ec3724b4b1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:25.101433 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90147c33-d292-454a-84b5-380401b8e7b7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:25.101698 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90147c33-d292-454a-84b5-380401b8e7b7 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:25.106534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 423/859] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:25.106534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90147c33-d292-454a-84b5-380401b8e7b7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:25.136538 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:25.139314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:25.209950 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0582d99e-61df-48c9-bd83-21ec3724b4b1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:25.217388 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90147c33-d292-454a-84b5-380401b8e7b7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:25.245255 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d958f-ecb4-41cb-9807-962e7c135748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:25.245907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d958f-ecb4-41cb-9807-962e7c135748 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:25.246184 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d958f-ecb4-41cb-9807-962e7c135748 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:25.246478 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d958f-ecb4-41cb-9807-962e7c135748 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:25.247376 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764d958f-ecb4-41cb-9807-962e7c135748 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:25.247799 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764d958f-ecb4-41cb-9807-962e7c135748 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:25.249216 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764d958f-ecb4-41cb-9807-962e7c135748 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:25.252299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e430fd4-bcc3-4eef-9391-f9d360aca2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:25.253087 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e430fd4-bcc3-4eef-9391-f9d360aca2ce None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:25.253374 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e430fd4-bcc3-4eef-9391-f9d360aca2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:25.253651 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e430fd4-bcc3-4eef-9391-f9d360aca2ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:25.266694 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 437/860] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:21:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:25.266694 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb4f7d-153b-43b7-900b-3381dc834b95 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:25.266694 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb4f7d-153b-43b7-900b-3381dc834b95 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:25.266694 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb4f7d-153b-43b7-900b-3381dc834b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:25.269523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e430fd4-bcc3-4eef-9391-f9d360aca2ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:25.271094 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e430fd4-bcc3-4eef-9391-f9d360aca2ce None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:25.280838 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d0bb4f7d-153b-43b7-900b-3381dc834b95 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:25.282281 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d0bb4f7d-153b-43b7-900b-3381dc834b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:25.290187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2e430fd4-bcc3-4eef-9391-f9d360aca2ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:25.290187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:25.290187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:25.290187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:25.290187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:25.290187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:25.290187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:25.290187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:25.290187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:25.290187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:25.290187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:25.290187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:25.290187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:25.290187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:25.290187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:25.291471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:25.291471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:25.291471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:25.291471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:25.291471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:25.291471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:25.291471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:25.291471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:25.291471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:25.291471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:25.291471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:25.291471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:25.291471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:25.291471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:25.291471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:25.291471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:25.291471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:25.292402 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:25.292402 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:25.292402 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:25.292402 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:25.292402 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 424/861] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:25.292678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e430fd4-bcc3-4eef-9391-f9d360aca2ce None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:25.366560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 438/862] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:25 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:25.367458 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb4f7d-153b-43b7-900b-3381dc834b95 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:25.390232 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:25.398490 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:25.415844 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e430fd4-bcc3-4eef-9391-f9d360aca2ce None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:25.452079 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca09cd-c737-4fe1-a385-95419b588f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:25.452734 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca09cd-c737-4fe1-a385-95419b588f1f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:25.453016 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca09cd-c737-4fe1-a385-95419b588f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:25.453290 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca09cd-c737-4fe1-a385-95419b588f1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:25.454123 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ca09cd-c737-4fe1-a385-95419b588f1f None admin] RBAC: Adding query filter params (role.id=3a7f4bf556344407a6d9540c89d13641, scope.project.id=f043ade9e4f748d4adc010ffb0a47a46, user.id=209b0df45f76404da880a9ae0c637d67) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:25.454450 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ca09cd-c737-4fe1-a385-95419b588f1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:25.455870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ca09cd-c737-4fe1-a385-95419b588f1f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:25.462094 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 425/863] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:21:25 2022] GET /identity/v3/role_assignments?user.id=209b0df45f76404da880a9ae0c637d67&scope.project.id=f043ade9e4f748d4adc010ffb0a47a46&role.id=3a7f4bf556344407a6d9540c89d13641 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:25.463394 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca09cd-c737-4fe1-a385-95419b588f1f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:25.493495 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb4f7d-153b-43b7-900b-3381dc834b95 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:25.536211 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a086c6a-27ca-4c72-825d-70cd91c5ac01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:25.537429 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a086c6a-27ca-4c72-825d-70cd91c5ac01 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:25.538023 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a086c6a-27ca-4c72-825d-70cd91c5ac01 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:25.538617 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a086c6a-27ca-4c72-825d-70cd91c5ac01 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:25.539596 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a086c6a-27ca-4c72-825d-70cd91c5ac01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:25.540156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a086c6a-27ca-4c72-825d-70cd91c5ac01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:25.542039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a086c6a-27ca-4c72-825d-70cd91c5ac01 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:25.556683 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 439/864] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:25.562214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a086c6a-27ca-4c72-825d-70cd91c5ac01 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:25.574800 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca09cd-c737-4fe1-a385-95419b588f1f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:25.613110 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898558b-55e5-4754-8809-93223205bc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:25.613859 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898558b-55e5-4754-8809-93223205bc1c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:25.614145 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898558b-55e5-4754-8809-93223205bc1c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:25.614448 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898558b-55e5-4754-8809-93223205bc1c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:25.615132 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c898558b-55e5-4754-8809-93223205bc1c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:25.616526 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c898558b-55e5-4754-8809-93223205bc1c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:25.630713 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c898558b-55e5-4754-8809-93223205bc1c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 12:21:25.630713 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:25.630713 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 12:21:25.630713 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 12:21:25.630713 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 12:21:25.630713 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 12:21:25.630713 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 12:21:25.630713 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:25.630713 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 12:21:25.630713 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:25.630713 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:21:25.630713 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:25.630713 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:25.630713 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:25.630713 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:25.630713 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:25.631736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:25.631736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:25.631736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:25.631736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:25.631736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:25.631736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:25.631736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:25.631736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 12:21:25.631736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 12:21:25.631736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 12:21:25.631736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 12:21:25.631736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:25.631736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:25.631736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:25.631736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:25.631736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:25.633017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:25.633017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:25.633017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:25.633017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:25.633017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:25.633017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:25.633017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:25.633017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:25.633017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:25.633017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:25.633017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:25.633017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 12:21:25.633017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:25.633017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 12:21:25.633017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:25.636914 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 426/865] 10.222.0.52 () {56 vars in 1132 bytes} [Wed Jun 29 12:21:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:25.637878 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898558b-55e5-4754-8809-93223205bc1c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:25.649364 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:25.672974 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:25.700687 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a086c6a-27ca-4c72-825d-70cd91c5ac01 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:25.739494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26527a2-e873-4979-b2b8-64ae94c54045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:25.740293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26527a2-e873-4979-b2b8-64ae94c54045 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:25.740606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26527a2-e873-4979-b2b8-64ae94c54045 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:25.740887 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26527a2-e873-4979-b2b8-64ae94c54045 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:25.749891 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26527a2-e873-4979-b2b8-64ae94c54045 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:25.751401 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26527a2-e873-4979-b2b8-64ae94c54045 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:25.766491 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a26527a2-e873-4979-b2b8-64ae94c54045 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:25.766491 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:25.766491 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:25.766491 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:25.766491 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:25.766491 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:25.766491 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:25.766491 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:25.766491 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:25.766491 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:25.766491 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:25.766491 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:25.766491 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:21:25.766491 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:21:25.766491 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:21:25.767403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:21:25.767403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:25.767403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:25.767403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:25.767403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:25.767403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:25.767403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:25.767403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:25.767403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:25.767403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:25.767403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:25.767403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:25.767403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:25.767403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:25.767403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:25.767403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:25.767403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:25.768294 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:21:25.768294 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:21:25.768294 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:21:25.768294 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:21:25.768294 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:21:25.768294 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:21:25.768294 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:25.768294 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:25.768722 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898558b-55e5-4754-8809-93223205bc1c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:25.768797 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 440/866] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:21:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:25.776541 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26527a2-e873-4979-b2b8-64ae94c54045 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:25.808824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68cf4d9-720e-4724-90ed-43162310c964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:25.809498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68cf4d9-720e-4724-90ed-43162310c964 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:25.809787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68cf4d9-720e-4724-90ed-43162310c964 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:25.810070 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68cf4d9-720e-4724-90ed-43162310c964 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:25.810817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68cf4d9-720e-4724-90ed-43162310c964 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:25.811128 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68cf4d9-720e-4724-90ed-43162310c964 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:25.812502 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68cf4d9-720e-4724-90ed-43162310c964 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:25.817836 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 427/867] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jun 29 12:21:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:25.823395 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68cf4d9-720e-4724-90ed-43162310c964 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:25.890149 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26527a2-e873-4979-b2b8-64ae94c54045 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:25.912911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:25.932295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9dd649-1568-4d9b-afc1-dc43fb6808a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:25.932992 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9dd649-1568-4d9b-afc1-dc43fb6808a3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:25.933247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9dd649-1568-4d9b-afc1-dc43fb6808a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:25.933528 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9dd649-1568-4d9b-afc1-dc43fb6808a3 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:25.934319 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9dd649-1568-4d9b-afc1-dc43fb6808a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:25.934665 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9dd649-1568-4d9b-afc1-dc43fb6808a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:25.936223 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9dd649-1568-4d9b-afc1-dc43fb6808a3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:25.953781 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68cf4d9-720e-4724-90ed-43162310c964 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:25.954437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 441/868] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:21:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:25.955331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9dd649-1568-4d9b-afc1-dc43fb6808a3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:25.958546 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:26.005513 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8568db-4be6-4c9a-ad79-8de08a30520c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:26.006198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8568db-4be6-4c9a-ad79-8de08a30520c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:26.006522 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8568db-4be6-4c9a-ad79-8de08a30520c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:26.006806 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8568db-4be6-4c9a-ad79-8de08a30520c None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:26.007945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8568db-4be6-4c9a-ad79-8de08a30520c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:26.008263 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8568db-4be6-4c9a-ad79-8de08a30520c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:26.010250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8568db-4be6-4c9a-ad79-8de08a30520c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:26.022467 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 428/869] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jun 29 12:21:25 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:26.022467 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800f07a-56ae-4428-b616-67f712216c8b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:26.022467 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800f07a-56ae-4428-b616-67f712216c8b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:26.022467 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800f07a-56ae-4428-b616-67f712216c8b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:26.022467 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 429/870] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:26.032288 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fc462-7006-449f-8e18-366ca6311d7e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:26.032910 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fc462-7006-449f-8e18-366ca6311d7e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:26.033413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998fc462-7006-449f-8e18-366ca6311d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:26.053140 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-998fc462-7006-449f-8e18-366ca6311d7e None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:26.054403 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-998fc462-7006-449f-8e18-366ca6311d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:26.081030 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9dd649-1568-4d9b-afc1-dc43fb6808a3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:26.120235 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76e72f-8595-4970-a515-0eea9d9f66b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:26.120913 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76e72f-8595-4970-a515-0eea9d9f66b1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:26.121203 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76e72f-8595-4970-a515-0eea9d9f66b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:26.121484 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76e72f-8595-4970-a515-0eea9d9f66b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:26.122334 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d76e72f-8595-4970-a515-0eea9d9f66b1 None admin] RBAC: Adding query filter params (role.id=8883c151055d4ba493f514ce1bb62e78, scope.project.id=f043ade9e4f748d4adc010ffb0a47a46, user.id=8c10852a728844c6ae0a48e3772d95b7) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:26.122685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d76e72f-8595-4970-a515-0eea9d9f66b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:26.124161 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d76e72f-8595-4970-a515-0eea9d9f66b1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:26.131378 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 442/871] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:21:25 2022] GET /identity/v3/role_assignments?user.id=8c10852a728844c6ae0a48e3772d95b7&scope.project.id=f043ade9e4f748d4adc010ffb0a47a46&role.id=8883c151055d4ba493f514ce1bb62e78 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:26.154594 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 430/872] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 12:21:26.179139 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76e72f-8595-4970-a515-0eea9d9f66b1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:26.258629 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:26.304231 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76e72f-8595-4970-a515-0eea9d9f66b1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:26.341991 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0305284-1cc7-48f1-994c-735ca21e0c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:26.343088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0305284-1cc7-48f1-994c-735ca21e0c0f None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:26.343592 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0305284-1cc7-48f1-994c-735ca21e0c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:26.344081 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0305284-1cc7-48f1-994c-735ca21e0c0f None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:26.345016 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0305284-1cc7-48f1-994c-735ca21e0c0f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:26.346618 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0305284-1cc7-48f1-994c-735ca21e0c0f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:26.367725 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 443/873] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jun 29 12:21:26 2022] POST /identity/v3/services => generated 251 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:21:27.662837 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dede1a-873f-493c-b4d1-efd41af1a2ae None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:27.663340 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dede1a-873f-493c-b4d1-efd41af1a2ae None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:27.663340 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dede1a-873f-493c-b4d1-efd41af1a2ae None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:27.664927 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 431/874] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:27.670556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1c56e-e5de-4e8e-a651-bbfbeb0aaf0a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:27.670961 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1c56e-e5de-4e8e-a651-bbfbeb0aaf0a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:27.671131 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1c56e-e5de-4e8e-a651-bbfbeb0aaf0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:27.687743 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-98b1c56e-e5de-4e8e-a651-bbfbeb0aaf0a None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:27.688902 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-98b1c56e-e5de-4e8e-a651-bbfbeb0aaf0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:27.761871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 444/875] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:27 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:27.784368 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dede1a-873f-493c-b4d1-efd41af1a2ae None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:27.854052 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:27.889514 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dede1a-873f-493c-b4d1-efd41af1a2ae None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:27.925290 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924c8227-5487-41b1-a9c5-cbb157c74707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:27.926005 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c8227-5487-41b1-a9c5-cbb157c74707 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:27.926285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c8227-5487-41b1-a9c5-cbb157c74707 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:27.926586 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c8227-5487-41b1-a9c5-cbb157c74707 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:27.927239 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924c8227-5487-41b1-a9c5-cbb157c74707 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:27.928599 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924c8227-5487-41b1-a9c5-cbb157c74707 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:27.941067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-924c8227-5487-41b1-a9c5-cbb157c74707 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 12:21:27.941067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:27.941067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 12:21:27.941067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 12:21:27.941067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 12:21:27.941067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 12:21:27.941067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 12:21:27.941067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:27.941067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 12:21:27.941067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:27.941067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:21:27.941067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:27.941067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:27.941067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:27.941067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:27.941067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:27.941067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:27.942299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:27.942299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:27.942299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:27.942299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:27.942299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:27.942299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:27.942299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 12:21:27.942299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 12:21:27.942299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 12:21:27.942299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 12:21:27.942299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:27.942299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:27.942299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:27.942299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:27.942299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:27.942299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:27.943358 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:27.943358 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:27.943358 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:27.943358 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:27.943358 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:27.943358 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:27.943358 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:27.943358 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:27.943358 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:27.943358 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:27.943358 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 12:21:27.943358 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:27.943358 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 12:21:27.943358 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:27.943358 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 432/876] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jun 29 12:21:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:27.948861 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1c56e-e5de-4e8e-a651-bbfbeb0aaf0a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:28.001938 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:28.051625 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1c56e-e5de-4e8e-a651-bbfbeb0aaf0a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:28.091041 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb63cf-b30e-443d-b1fe-04cddf309df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:28.092023 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb63cf-b30e-443d-b1fe-04cddf309df6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:28.092518 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb63cf-b30e-443d-b1fe-04cddf309df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:28.092977 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb63cf-b30e-443d-b1fe-04cddf309df6 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:28.093872 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb63cf-b30e-443d-b1fe-04cddf309df6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:28.094359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb63cf-b30e-443d-b1fe-04cddf309df6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:28.095932 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb63cf-b30e-443d-b1fe-04cddf309df6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:28.101586 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 445/877] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jun 29 12:21:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:28.107471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924c8227-5487-41b1-a9c5-cbb157c74707 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:28.135866 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe44d14-1c9f-44f3-8753-eec10a37159a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:28.136600 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe44d14-1c9f-44f3-8753-eec10a37159a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:28.137308 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe44d14-1c9f-44f3-8753-eec10a37159a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:28.139433 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 446/878] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:28.144619 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7514e3-bf63-4408-a041-8b3cbd61d64e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:28.147154 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7514e3-bf63-4408-a041-8b3cbd61d64e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:28.147154 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7514e3-bf63-4408-a041-8b3cbd61d64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:28.163962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0e7514e3-bf63-4408-a041-8b3cbd61d64e None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:28.165022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0e7514e3-bf63-4408-a041-8b3cbd61d64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:28.217013 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924c8227-5487-41b1-a9c5-cbb157c74707 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:28.246578 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:28.254474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 447/879] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:28 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:28.262585 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13d6c1-1577-4783-a59b-2590f3afbff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:28.263519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13d6c1-1577-4783-a59b-2590f3afbff4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:28.263981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13d6c1-1577-4783-a59b-2590f3afbff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:28.264448 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13d6c1-1577-4783-a59b-2590f3afbff4 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:28.265339 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13d6c1-1577-4783-a59b-2590f3afbff4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:28.265833 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13d6c1-1577-4783-a59b-2590f3afbff4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:28.267402 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13d6c1-1577-4783-a59b-2590f3afbff4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:28.273020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 433/880] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jun 29 12:21:28 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:28.279586 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7514e3-bf63-4408-a041-8b3cbd61d64e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:28.336246 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:28.409259 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7514e3-bf63-4408-a041-8b3cbd61d64e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:28.447205 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd5f64d-c07a-4302-8077-8c14caea3e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:28.447886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5f64d-c07a-4302-8077-8c14caea3e73 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:28.448167 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5f64d-c07a-4302-8077-8c14caea3e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:28.448442 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd5f64d-c07a-4302-8077-8c14caea3e73 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:28.449177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd5f64d-c07a-4302-8077-8c14caea3e73 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:28.450612 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd5f64d-c07a-4302-8077-8c14caea3e73 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:28.472812 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 448/881] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jun 29 12:21:28 2022] POST /identity/v3/services => generated 269 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:21:28.474393 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5eb234-edf4-473b-9891-161d30ca5efe None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:28.474718 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5eb234-edf4-473b-9891-161d30ca5efe None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:28.475009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5eb234-edf4-473b-9891-161d30ca5efe None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:28.476748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 434/882] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:28.483266 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40b5ce-9766-44ba-9e8b-9e585a14e897 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:28.483570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40b5ce-9766-44ba-9e8b-9e585a14e897 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:28.483836 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40b5ce-9766-44ba-9e8b-9e585a14e897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:28.496259 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c3353-c21f-4457-90cc-1c5aeb01cccb None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:28.496579 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c3353-c21f-4457-90cc-1c5aeb01cccb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:28.496809 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51c3353-c21f-4457-90cc-1c5aeb01cccb None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:28.498386 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 435/883] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:28.504704 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3b40b5ce-9766-44ba-9e8b-9e585a14e897 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:28.505275 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29d70e-5c7a-4572-bc55-cbf105c6b5d6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:28.505560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29d70e-5c7a-4572-bc55-cbf105c6b5d6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:28.505853 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd29d70e-5c7a-4572-bc55-cbf105c6b5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:28.505961 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3b40b5ce-9766-44ba-9e8b-9e585a14e897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:28.529913 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bd29d70e-5c7a-4572-bc55-cbf105c6b5d6 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:28.531099 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bd29d70e-5c7a-4572-bc55-cbf105c6b5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:28.614919 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 449/884] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:28 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:28.624725 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 436/885] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:28 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:28.638477 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b40b5ce-9766-44ba-9e8b-9e585a14e897 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:28.645365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd29d70e-5c7a-4572-bc55-cbf105c6b5d6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:28.673235 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:28.762344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b40b5ce-9766-44ba-9e8b-9e585a14e897 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:28.776579 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd29d70e-5c7a-4572-bc55-cbf105c6b5d6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:28.802917 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce52a6-1deb-4842-b1a3-1f11443cdf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:28.803727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce52a6-1deb-4842-b1a3-1f11443cdf3f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:28.804022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce52a6-1deb-4842-b1a3-1f11443cdf3f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:28.804306 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce52a6-1deb-4842-b1a3-1f11443cdf3f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:28.805274 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:28.806569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ce52a6-1deb-4842-b1a3-1f11443cdf3f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:28.808331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ce52a6-1deb-4842-b1a3-1f11443cdf3f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:28.822178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf378e9-739c-4e5e-81b9-a3f4308c16c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:28.823364 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf378e9-739c-4e5e-81b9-a3f4308c16c7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:28.823829 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf378e9-739c-4e5e-81b9-a3f4308c16c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:28.824304 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf378e9-739c-4e5e-81b9-a3f4308c16c7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:28.824645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b3ce52a6-1deb-4842-b1a3-1f11443cdf3f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 12:21:28.824645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:28.824645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 12:21:28.824645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 12:21:28.824645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 12:21:28.824645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 12:21:28.824645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 12:21:28.824645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:28.824645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 12:21:28.824645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:28.824645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:21:28.824645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:28.824645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:28.824645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:28.824645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:28.824645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:28.824645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:28.825853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:28.825853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:28.825853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:28.825853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:28.825853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:28.825853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:28.825853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 12:21:28.825853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 12:21:28.825853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 12:21:28.825853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 12:21:28.825853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:28.825853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:28.825853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:28.825853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:28.825853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:28.825853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:28.826876 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:28.826876 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:28.826876 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:28.826876 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:28.826876 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:28.826876 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:28.826876 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:28.826876 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:28.826876 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:28.826876 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:28.826876 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 12:21:28.826876 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:28.826876 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 12:21:28.826876 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:28.826876 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 450/886] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 12:21:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:28.831993 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce52a6-1deb-4842-b1a3-1f11443cdf3f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:28.842623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf378e9-739c-4e5e-81b9-a3f4308c16c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:21:28.851049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1bf378e9-739c-4e5e-81b9-a3f4308c16c7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 12:21:28.851049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:28.851049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:28.851049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:28.851049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:28.851049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:28.851049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:28.851049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:28.851049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:28.851049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:28.851049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:28.851049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:28.851049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:21:28.851049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:21:28.851049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:21:28.851049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:21:28.852000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:21:28.852000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:21:28.852000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:21:28.852000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:21:28.852000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:28.852000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:28.852000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:28.852000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:28.852000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:28.852000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:28.852000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:28.852000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:28.852000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:28.852000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:28.852000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:28.852000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:28.853498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:28.853498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:28.853498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:28.853498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:28.853498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:21:28.853498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:21:28.853498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:21:28.853498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:21:28.853498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:21:28.853498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:21:28.853498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 12:21:28.853498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:28.856551 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 437/887] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Jun 29 12:21:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:28.866178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf378e9-739c-4e5e-81b9-a3f4308c16c7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:28.958456 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:28.966792 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce52a6-1deb-4842-b1a3-1f11443cdf3f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:28.980970 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf378e9-739c-4e5e-81b9-a3f4308c16c7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:29.013784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c3899-c54c-48f9-a41f-ce2cd078e709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:29.014495 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c3899-c54c-48f9-a41f-ce2cd078e709 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:29.015024 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c3899-c54c-48f9-a41f-ce2cd078e709 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:29.015474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c3899-c54c-48f9-a41f-ce2cd078e709 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:29.016308 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c3899-c54c-48f9-a41f-ce2cd078e709 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:29.016691 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c3899-c54c-48f9-a41f-ce2cd078e709 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:29.018134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c3899-c54c-48f9-a41f-ce2cd078e709 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:29.019840 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ebc76a-3db9-42ca-a864-ee5fa84257b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:29.020783 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebc76a-3db9-42ca-a864-ee5fa84257b1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:29.021279 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebc76a-3db9-42ca-a864-ee5fa84257b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:29.022042 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebc76a-3db9-42ca-a864-ee5fa84257b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:29.023029 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ebc76a-3db9-42ca-a864-ee5fa84257b1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:29.023535 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ebc76a-3db9-42ca-a864-ee5fa84257b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:29.025723 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ebc76a-3db9-42ca-a864-ee5fa84257b1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:29.029003 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 451/888] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 12:21:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:29.030322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c3899-c54c-48f9-a41f-ce2cd078e709 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:29.042280 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 438/889] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Jun 29 12:21:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:29.043646 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ebc76a-3db9-42ca-a864-ee5fa84257b1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:29.096262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:29.156541 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c3899-c54c-48f9-a41f-ce2cd078e709 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:29.174202 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ebc76a-3db9-42ca-a864-ee5fa84257b1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:29.193772 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca4f46-8864-46eb-922a-fb5b98a34556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:29.194825 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca4f46-8864-46eb-922a-fb5b98a34556 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:29.195311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca4f46-8864-46eb-922a-fb5b98a34556 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:29.195785 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca4f46-8864-46eb-922a-fb5b98a34556 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:29.196819 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca4f46-8864-46eb-922a-fb5b98a34556 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:29.197401 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca4f46-8864-46eb-922a-fb5b98a34556 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:29.199046 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca4f46-8864-46eb-922a-fb5b98a34556 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:29.204556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 452/890] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 12:21:29 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:29.210289 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca4f46-8864-46eb-922a-fb5b98a34556 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:29.219437 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639521cf-3d78-4286-9f55-6dce7752c506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:29.220217 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639521cf-3d78-4286-9f55-6dce7752c506 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:29.220510 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639521cf-3d78-4286-9f55-6dce7752c506 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:29.220801 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639521cf-3d78-4286-9f55-6dce7752c506 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:29.235189 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639521cf-3d78-4286-9f55-6dce7752c506 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:29.237098 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639521cf-3d78-4286-9f55-6dce7752c506 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:29.246829 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:29.263142 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-639521cf-3d78-4286-9f55-6dce7752c506 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:29.263142 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:29.263142 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:29.263142 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:29.263142 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:29.263142 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:29.263142 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:29.263142 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:29.263142 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:29.263142 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:29.263142 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:29.263142 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:29.263142 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:29.263142 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:29.263142 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:29.264092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:29.264092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:29.264092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:29.264092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:29.264092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:29.264092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:29.264092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:29.264092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:29.264092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:29.264092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:29.264092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:29.264092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:29.264092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:29.264092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:29.264092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:29.264092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:29.264092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:29.264997 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:29.264997 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:29.264997 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:29.264997 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:29.269279 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 439/891] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:29.270139 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639521cf-3d78-4286-9f55-6dce7752c506 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:29.318898 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca4f46-8864-46eb-922a-fb5b98a34556 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:29.357580 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ee17d-d4ff-4c15-acc4-1eeb7f597107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:29.358252 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ee17d-d4ff-4c15-acc4-1eeb7f597107 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:29.358572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ee17d-d4ff-4c15-acc4-1eeb7f597107 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:29.358857 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06ee17d-d4ff-4c15-acc4-1eeb7f597107 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:29.359760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ee17d-d4ff-4c15-acc4-1eeb7f597107 None admin] RBAC: Adding query filter params (interface=public, service_id=99b2d251a00e4dd4acdfa37ad117dc57, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:29.360085 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ee17d-d4ff-4c15-acc4-1eeb7f597107 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:29.361564 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06ee17d-d4ff-4c15-acc4-1eeb7f597107 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:29.372351 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 453/892] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Jun 29 12:21:29 2022] GET /identity/v3/endpoints?service_id=99b2d251a00e4dd4acdfa37ad117dc57&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:29.373148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ee17d-d4ff-4c15-acc4-1eeb7f597107 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:29.378748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:29.386828 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639521cf-3d78-4286-9f55-6dce7752c506 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:29.425595 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d7e23e-93f6-4b8a-9362-2c2677e3e1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:29.426368 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7e23e-93f6-4b8a-9362-2c2677e3e1d6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:29.426683 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7e23e-93f6-4b8a-9362-2c2677e3e1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:29.426962 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d7e23e-93f6-4b8a-9362-2c2677e3e1d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:29.427683 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d7e23e-93f6-4b8a-9362-2c2677e3e1d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:29.427988 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d7e23e-93f6-4b8a-9362-2c2677e3e1d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:29.429541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d7e23e-93f6-4b8a-9362-2c2677e3e1d6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:29.447159 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 440/893] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:29.448944 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d7e23e-93f6-4b8a-9362-2c2677e3e1d6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:29.470654 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:29.484302 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06ee17d-d4ff-4c15-acc4-1eeb7f597107 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:29.521961 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da990759-b2e0-4ebe-b6c8-3b3977f1e41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:29.522762 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da990759-b2e0-4ebe-b6c8-3b3977f1e41e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:29.523204 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da990759-b2e0-4ebe-b6c8-3b3977f1e41e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:29.523554 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da990759-b2e0-4ebe-b6c8-3b3977f1e41e None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:29.524316 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da990759-b2e0-4ebe-b6c8-3b3977f1e41e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:29.525861 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da990759-b2e0-4ebe-b6c8-3b3977f1e41e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:29.538953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 454/894] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Jun 29 12:21:29 2022] GET /identity/v3/services => generated 1268 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:21:29.560799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d7e23e-93f6-4b8a-9362-2c2677e3e1d6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:29.602064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342277c-a2a0-4cde-9c8d-e634a7ef811e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:29.604391 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342277c-a2a0-4cde-9c8d-e634a7ef811e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:29.605139 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342277c-a2a0-4cde-9c8d-e634a7ef811e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:29.605910 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342277c-a2a0-4cde-9c8d-e634a7ef811e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:29.614879 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342277c-a2a0-4cde-9c8d-e634a7ef811e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:29.616648 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342277c-a2a0-4cde-9c8d-e634a7ef811e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:29.633534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8342277c-a2a0-4cde-9c8d-e634a7ef811e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 12:21:29.633534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:29.633534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:29.633534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:29.633534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:29.633534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:29.633534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:29.633534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:29.633534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:29.633534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:29.633534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:29.633534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:29.633534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:21:29.633534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:21:29.633534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:21:29.633534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:21:29.635009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:29.635009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:29.635009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:21:29.635009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:29.635009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:21:29.635009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:29.635009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:29.635009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:29.635009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:29.635009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:29.635009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:29.635009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:29.635009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:29.635009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:29.635009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:29.635009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:29.635009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:29.635009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:29.636046 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:29.636046 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:29.636046 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:21:29.636046 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:21:29.636046 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:21:29.636046 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:21:29.636046 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:21:29.636046 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:21:29.636046 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 12:21:29.636046 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:29.638515 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 441/895] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jun 29 12:21:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:29.650467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da990759-b2e0-4ebe-b6c8-3b3977f1e41e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:29.760869 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da990759-b2e0-4ebe-b6c8-3b3977f1e41e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:29.796318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373951f-6074-41fb-9ad3-4c5d0c023fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:29.796919 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373951f-6074-41fb-9ad3-4c5d0c023fea None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:29.797207 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373951f-6074-41fb-9ad3-4c5d0c023fea None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:29.797489 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373951f-6074-41fb-9ad3-4c5d0c023fea None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:29.798328 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373951f-6074-41fb-9ad3-4c5d0c023fea None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:29.798675 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373951f-6074-41fb-9ad3-4c5d0c023fea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:29.800129 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373951f-6074-41fb-9ad3-4c5d0c023fea None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:29.818166 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 455/896] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Jun 29 12:21:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:29.823928 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342277c-a2a0-4cde-9c8d-e634a7ef811e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:29.841638 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:29.930903 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342277c-a2a0-4cde-9c8d-e634a7ef811e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:29.965888 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb880cc-b592-43c8-88a4-92935079374e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:29.966932 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb880cc-b592-43c8-88a4-92935079374e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:29.967431 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb880cc-b592-43c8-88a4-92935079374e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:29.967913 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb880cc-b592-43c8-88a4-92935079374e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:30.038044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb880cc-b592-43c8-88a4-92935079374e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:30.039531 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb880cc-b592-43c8-88a4-92935079374e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:30.055100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:30.146821 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 442/897] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:21:29 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 12:21:30.152730 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373951f-6074-41fb-9ad3-4c5d0c023fea None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:30.165684 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:30.257982 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373951f-6074-41fb-9ad3-4c5d0c023fea None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:30.294299 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbc8e76-3dfe-4a15-8c04-d50ad20ad9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:30.295585 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc8e76-3dfe-4a15-8c04-d50ad20ad9df None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:30.296095 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc8e76-3dfe-4a15-8c04-d50ad20ad9df None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:30.296572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbc8e76-3dfe-4a15-8c04-d50ad20ad9df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:30.307837 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc8e76-3dfe-4a15-8c04-d50ad20ad9df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:30.309613 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbc8e76-3dfe-4a15-8c04-d50ad20ad9df None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:30.327883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-adbc8e76-3dfe-4a15-8c04-d50ad20ad9df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:30.327883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:30.327883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:30.327883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:30.327883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:30.327883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:30.327883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:30.327883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:30.327883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:30.327883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:30.327883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:30.327883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:30.327883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:30.327883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:30.327883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:30.328856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:30.328856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:30.328856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:30.328856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:30.328856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:30.328856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:30.328856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:30.328856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:30.328856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:30.328856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:30.328856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:30.328856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:30.328856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:30.328856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:30.328856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:30.328856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:30.328856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:30.329751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:30.329751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:30.329751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:30.329751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:30.332110 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 456/898] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:30.338440 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb880cc-b592-43c8-88a4-92935079374e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:30.435401 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0eec76-684a-4b45-8a26-8e46f239f865 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:30.436102 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0eec76-684a-4b45-8a26-8e46f239f865 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:30.436596 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0eec76-684a-4b45-8a26-8e46f239f865 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:30.438590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 457/899] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:30.443212 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3dbe2-ef69-4223-86b0-336428e12f75 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:30.443730 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3dbe2-ef69-4223-86b0-336428e12f75 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:30.444210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3dbe2-ef69-4223-86b0-336428e12f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:30.444677 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb880cc-b592-43c8-88a4-92935079374e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:30.459915 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:30.463962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-00e3dbe2-ef69-4223-86b0-336428e12f75 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:30.466333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-00e3dbe2-ef69-4223-86b0-336428e12f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:30.481827 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83397df1-c189-453f-92b7-4c745c17d3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:30.482583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83397df1-c189-453f-92b7-4c745c17d3fe None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:30.482806 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83397df1-c189-453f-92b7-4c745c17d3fe None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:30.483082 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83397df1-c189-453f-92b7-4c745c17d3fe None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:30.483803 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83397df1-c189-453f-92b7-4c745c17d3fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:30.484108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83397df1-c189-453f-92b7-4c745c17d3fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:30.485491 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83397df1-c189-453f-92b7-4c745c17d3fe None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:30.498163 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 443/900] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:30.503884 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83397df1-c189-453f-92b7-4c745c17d3fe None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:30.540639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:30.553766 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 458/901] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:30 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:30.578041 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e3dbe2-ef69-4223-86b0-336428e12f75 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:30.612521 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83397df1-c189-453f-92b7-4c745c17d3fe None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:30.648788 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669a5d9-e7ee-43b2-bb70-e09c0c9d81d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:30.649604 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669a5d9-e7ee-43b2-bb70-e09c0c9d81d8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:30.650406 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669a5d9-e7ee-43b2-bb70-e09c0c9d81d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:30.650710 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669a5d9-e7ee-43b2-bb70-e09c0c9d81d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:30.657465 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669a5d9-e7ee-43b2-bb70-e09c0c9d81d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:30.659614 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669a5d9-e7ee-43b2-bb70-e09c0c9d81d8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:30.673627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0669a5d9-e7ee-43b2-bb70-e09c0c9d81d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:30.673627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:30.673627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:30.673627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:30.673627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:30.673627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:30.673627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:30.673627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:30.673627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:30.673627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:30.673627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:30.673627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:30.673627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:21:30.673627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:21:30.673627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:21:30.674626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:21:30.674626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:30.674626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:30.674626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:30.674626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:30.674626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:30.674626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:30.674626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:30.674626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:30.674626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:30.674626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:30.674626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:30.674626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:30.674626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:30.674626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:30.674626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:30.674626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:30.675524 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:21:30.675524 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:21:30.675524 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:21:30.675524 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:21:30.675524 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:21:30.675524 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:21:30.675524 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:30.675524 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:30.679654 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 444/902] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:21:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:30.680491 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669a5d9-e7ee-43b2-bb70-e09c0c9d81d8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:30.694986 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e3dbe2-ef69-4223-86b0-336428e12f75 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:30.714677 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:30.736366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b27d72-15a7-4420-9426-e63298c20195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:30.737047 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b27d72-15a7-4420-9426-e63298c20195 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:30.737330 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b27d72-15a7-4420-9426-e63298c20195 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:30.737608 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b27d72-15a7-4420-9426-e63298c20195 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:30.738363 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b27d72-15a7-4420-9426-e63298c20195 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:30.739833 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b27d72-15a7-4420-9426-e63298c20195 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:30.757803 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 459/903] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jun 29 12:21:30 2022] POST /identity/v3/services => generated 241 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:21:30.763734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda9473-cc0e-4468-891b-f48277ad3f3f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:30.764378 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda9473-cc0e-4468-891b-f48277ad3f3f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:30.764378 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afda9473-cc0e-4468-891b-f48277ad3f3f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:30.769965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 460/904] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:21:30.771721 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b67914-8240-43fa-b506-4360dbceb0d3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:30.772015 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b67914-8240-43fa-b506-4360dbceb0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:30.772302 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b67914-8240-43fa-b506-4360dbceb0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:30.791110 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-48b67914-8240-43fa-b506-4360dbceb0d3 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:30.792895 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-48b67914-8240-43fa-b506-4360dbceb0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:30.805381 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669a5d9-e7ee-43b2-bb70-e09c0c9d81d8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:30.857305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407b568-d5d6-4ab6-9803-f6c07bd5b01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:30.858781 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407b568-d5d6-4ab6-9803-f6c07bd5b01a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:30.859286 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407b568-d5d6-4ab6-9803-f6c07bd5b01a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:30.859862 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407b568-d5d6-4ab6-9803-f6c07bd5b01a None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:30.860683 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6407b568-d5d6-4ab6-9803-f6c07bd5b01a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:30.861368 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:30.861564 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6407b568-d5d6-4ab6-9803-f6c07bd5b01a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:30.863970 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6407b568-d5d6-4ab6-9803-f6c07bd5b01a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:30.878745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 445/905] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:21:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:30.884925 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407b568-d5d6-4ab6-9803-f6c07bd5b01a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:30.899223 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 461/906] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:30.926226 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b67914-8240-43fa-b506-4360dbceb0d3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:31.004451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:31.016553 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407b568-d5d6-4ab6-9803-f6c07bd5b01a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:31.053803 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51627f40-7c37-47da-b2f6-e062a8e31010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:31.054527 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51627f40-7c37-47da-b2f6-e062a8e31010 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:31.054801 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51627f40-7c37-47da-b2f6-e062a8e31010 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:31.055076 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51627f40-7c37-47da-b2f6-e062a8e31010 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:31.056019 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51627f40-7c37-47da-b2f6-e062a8e31010 None admin] RBAC: Adding query filter params (role.id=7dedcc77e86a4fee94bf84105c0ffd71, scope.project.id=f043ade9e4f748d4adc010ffb0a47a46, user.id=8c10852a728844c6ae0a48e3772d95b7) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:31.056301 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51627f40-7c37-47da-b2f6-e062a8e31010 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:31.057901 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51627f40-7c37-47da-b2f6-e062a8e31010 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:31.068111 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 446/907] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:21:30 2022] GET /identity/v3/role_assignments?user.id=8c10852a728844c6ae0a48e3772d95b7&scope.project.id=f043ade9e4f748d4adc010ffb0a47a46&role.id=7dedcc77e86a4fee94bf84105c0ffd71 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:31.078391 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b67914-8240-43fa-b506-4360dbceb0d3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:31.121714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f1f9d-20ed-454a-a4ab-87bef186d237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:31.122939 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f1f9d-20ed-454a-a4ab-87bef186d237 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:31.123447 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f1f9d-20ed-454a-a4ab-87bef186d237 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:31.123944 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f1f9d-20ed-454a-a4ab-87bef186d237 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:31.124833 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f1f9d-20ed-454a-a4ab-87bef186d237 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:31.126536 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f1f9d-20ed-454a-a4ab-87bef186d237 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:31.141505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-4a2f1f9d-20ed-454a-a4ab-87bef186d237 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 12:21:31.141505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:31.141505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 12:21:31.141505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 12:21:31.141505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 12:21:31.141505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 12:21:31.141505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 12:21:31.141505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:31.141505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 12:21:31.141505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:31.141505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:21:31.141505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:31.141505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:31.141505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:31.141505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:31.141505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:31.142606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:31.142606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:31.142606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:31.142606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:31.142606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:31.142606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:31.142606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:31.142606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 12:21:31.142606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 12:21:31.142606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 12:21:31.142606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 12:21:31.142606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:31.142606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:31.142606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:31.142606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:31.142606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:31.143459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:31.143459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:31.143459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:31.143459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:31.143459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:31.143459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:31.143459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:31.143459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:31.143459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:31.143459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:31.143459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:31.143459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 12:21:31.143459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:31.143459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 12:21:31.143459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:31.151259 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 462/908] 10.222.0.52 () {56 vars in 1132 bytes} [Wed Jun 29 12:21:30 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:31.154598 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51627f40-7c37-47da-b2f6-e062a8e31010 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:31.276735 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51627f40-7c37-47da-b2f6-e062a8e31010 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:31.314008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6a586-e9c5-4f01-97e5-aeb3ebe43e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:31.315119 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6a586-e9c5-4f01-97e5-aeb3ebe43e7f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:31.315610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6a586-e9c5-4f01-97e5-aeb3ebe43e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:31.316087 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6a586-e9c5-4f01-97e5-aeb3ebe43e7f None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:31.316997 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6a586-e9c5-4f01-97e5-aeb3ebe43e7f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:31.317527 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6a586-e9c5-4f01-97e5-aeb3ebe43e7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:31.319206 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6a586-e9c5-4f01-97e5-aeb3ebe43e7f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:31.325279 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 447/909] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jun 29 12:21:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:31.331145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f1f9d-20ed-454a-a4ab-87bef186d237 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:31.384781 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:31.442266 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f1f9d-20ed-454a-a4ab-87bef186d237 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:31.478914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb1712-92f1-4d09-a01f-de60dd195965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:31.479593 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb1712-92f1-4d09-a01f-de60dd195965 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:31.479882 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb1712-92f1-4d09-a01f-de60dd195965 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:31.480163 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb1712-92f1-4d09-a01f-de60dd195965 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:31.480878 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb1712-92f1-4d09-a01f-de60dd195965 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:31.481187 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb1712-92f1-4d09-a01f-de60dd195965 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:31.482588 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb1712-92f1-4d09-a01f-de60dd195965 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:31.488122 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 463/910] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jun 29 12:21:31 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:31.494468 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6a586-e9c5-4f01-97e5-aeb3ebe43e7f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:31.516076 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:31.609256 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6a586-e9c5-4f01-97e5-aeb3ebe43e7f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:31.645175 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f2eb8a-fa34-45b8-9e74-b59df2963ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:31.645757 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2eb8a-fa34-45b8-9e74-b59df2963ebb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:31.646040 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2eb8a-fa34-45b8-9e74-b59df2963ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:31.646313 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f2eb8a-fa34-45b8-9e74-b59df2963ebb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:31.647381 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2eb8a-fa34-45b8-9e74-b59df2963ebb None admin] RBAC: Adding query filter params (interface=public, service_id=37e2592485e94f06bc12bf53310b6be9, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:31.647692 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2eb8a-fa34-45b8-9e74-b59df2963ebb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:31.649048 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f2eb8a-fa34-45b8-9e74-b59df2963ebb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:31.655288 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 448/911] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Jun 29 12:21:31 2022] GET /identity/v3/endpoints?service_id=37e2592485e94f06bc12bf53310b6be9&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:31.663538 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb1712-92f1-4d09-a01f-de60dd195965 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:31.715166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0e09b-8c38-4711-a31a-247c130c3aef None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:31.715505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0e09b-8c38-4711-a31a-247c130c3aef None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:31.715727 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0e09b-8c38-4711-a31a-247c130c3aef None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:31.717322 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 449/912] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:21:31.722154 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d93e70-fef5-4b4c-a567-7e21d1597328 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:31.722451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d93e70-fef5-4b4c-a567-7e21d1597328 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:31.722740 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d93e70-fef5-4b4c-a567-7e21d1597328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:31.741410 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f9d93e70-fef5-4b4c-a567-7e21d1597328 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:31.742622 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f9d93e70-fef5-4b4c-a567-7e21d1597328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:31.770441 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb1712-92f1-4d09-a01f-de60dd195965 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:31.782771 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:31.807430 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b60eaf-848e-49e0-aae4-d645ec88ae34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:31.808502 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b60eaf-848e-49e0-aae4-d645ec88ae34 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:31.808831 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b60eaf-848e-49e0-aae4-d645ec88ae34 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:31.809387 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b60eaf-848e-49e0-aae4-d645ec88ae34 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:31.810149 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b60eaf-848e-49e0-aae4-d645ec88ae34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:31.811931 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b60eaf-848e-49e0-aae4-d645ec88ae34 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:31.823615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 450/913] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:31.823888 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 464/914] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Jun 29 12:21:31 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:21:31.839684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d93e70-fef5-4b4c-a567-7e21d1597328 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:31.873421 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:31.951497 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d93e70-fef5-4b4c-a567-7e21d1597328 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:31.997004 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafcada7-3b89-4808-955d-60eb8cf1488c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:31.998236 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcada7-3b89-4808-955d-60eb8cf1488c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:31.998637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcada7-3b89-4808-955d-60eb8cf1488c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:31.998920 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcada7-3b89-4808-955d-60eb8cf1488c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:31.999603 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafcada7-3b89-4808-955d-60eb8cf1488c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:32.001017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafcada7-3b89-4808-955d-60eb8cf1488c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:32.014051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-dafcada7-3b89-4808-955d-60eb8cf1488c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 12:21:32.014051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:32.014051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 12:21:32.014051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 12:21:32.014051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 12:21:32.014051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 12:21:32.014051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 12:21:32.014051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:32.014051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 12:21:32.014051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:32.014051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:21:32.014051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:32.014051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:32.014051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:32.014051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:32.014051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:32.014051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:32.015220 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:32.015220 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:32.015220 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:32.015220 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:32.015220 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:32.015220 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:32.015220 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 12:21:32.015220 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 12:21:32.015220 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 12:21:32.015220 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 12:21:32.015220 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:32.015220 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:32.015220 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:32.015220 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:32.015220 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:32.015220 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:32.016080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:32.016080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:32.016080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:32.016080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:32.016080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:32.016080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:32.016080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:32.016080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:32.016080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:32.016080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:32.016080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 12:21:32.016080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:32.016080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 29 12:21:32.016080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:32.016080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 451/915] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 12:21:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:32.023844 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b60eaf-848e-49e0-aae4-d645ec88ae34 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:32.128337 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b60eaf-848e-49e0-aae4-d645ec88ae34 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:32.163315 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40dd358-2b0c-4699-af19-11c0f1be9e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:32.164354 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40dd358-2b0c-4699-af19-11c0f1be9e18 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:32.164837 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40dd358-2b0c-4699-af19-11c0f1be9e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:32.165875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40dd358-2b0c-4699-af19-11c0f1be9e18 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:32.166805 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40dd358-2b0c-4699-af19-11c0f1be9e18 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:32.167310 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40dd358-2b0c-4699-af19-11c0f1be9e18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:32.168900 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40dd358-2b0c-4699-af19-11c0f1be9e18 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:32.178531 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 465/916] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 12:21:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:32.180360 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafcada7-3b89-4808-955d-60eb8cf1488c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:32.266681 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:32.291093 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafcada7-3b89-4808-955d-60eb8cf1488c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:32.325854 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0ce98-f731-41fb-9c35-3f61f27f6874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:32.326546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0ce98-f731-41fb-9c35-3f61f27f6874 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:32.326833 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0ce98-f731-41fb-9c35-3f61f27f6874 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:32.327111 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0ce98-f731-41fb-9c35-3f61f27f6874 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:32.327819 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0ce98-f731-41fb-9c35-3f61f27f6874 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:32.328121 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0ce98-f731-41fb-9c35-3f61f27f6874 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:32.329586 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0ce98-f731-41fb-9c35-3f61f27f6874 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:32.334811 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 452/917] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jun 29 12:21:32 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:32.340732 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40dd358-2b0c-4699-af19-11c0f1be9e18 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:32.389097 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:32.456443 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40dd358-2b0c-4699-af19-11c0f1be9e18 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:32.491364 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c9071-551b-4fda-a4d2-90a18ae0276c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:32.492266 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c9071-551b-4fda-a4d2-90a18ae0276c None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:32.492703 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c9071-551b-4fda-a4d2-90a18ae0276c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:32.493151 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c9071-551b-4fda-a4d2-90a18ae0276c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:32.494302 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8c9071-551b-4fda-a4d2-90a18ae0276c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:32.495854 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8c9071-551b-4fda-a4d2-90a18ae0276c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:32.518445 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 466/918] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Jun 29 12:21:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:21:32.990148 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6a7d9-15e4-49be-b034-30a3d0aecf6e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:32.990668 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6a7d9-15e4-49be-b034-30a3d0aecf6e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:32.990668 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6a7d9-15e4-49be-b034-30a3d0aecf6e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:32.992296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 453/919] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:32.998730 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e94165-471f-4db4-ac48-2e410cedcd77 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:32.999164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e94165-471f-4db4-ac48-2e410cedcd77 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:32.999361 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e94165-471f-4db4-ac48-2e410cedcd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:33.016295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b8e94165-471f-4db4-ac48-2e410cedcd77 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:33.017493 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b8e94165-471f-4db4-ac48-2e410cedcd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:33.095037 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 467/920] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:33.118971 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6a7d9-15e4-49be-b034-30a3d0aecf6e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:33.163179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfacce6-078b-4f2c-b53b-6e0db42101dc None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:33.163510 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfacce6-078b-4f2c-b53b-6e0db42101dc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:33.163745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfacce6-078b-4f2c-b53b-6e0db42101dc None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:33.165947 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 468/921] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:33.171492 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9c15d-980c-49c0-98d7-2bb9b25dba9a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:33.171779 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9c15d-980c-49c0-98d7-2bb9b25dba9a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:33.172058 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9c15d-980c-49c0-98d7-2bb9b25dba9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:33.193216 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-70e9c15d-980c-49c0-98d7-2bb9b25dba9a None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:33.194622 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-70e9c15d-980c-49c0-98d7-2bb9b25dba9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:33.215172 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:33.246578 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6a7d9-15e4-49be-b034-30a3d0aecf6e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:33.276939 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 469/922] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:33.287283 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a547a227-81da-472c-850b-1a368c53a54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:33.288101 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547a227-81da-472c-850b-1a368c53a54f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:33.288395 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547a227-81da-472c-850b-1a368c53a54f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:33.288674 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a547a227-81da-472c-850b-1a368c53a54f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:33.289393 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a547a227-81da-472c-850b-1a368c53a54f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:33.290976 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a547a227-81da-472c-850b-1a368c53a54f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:33.298570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9c15d-980c-49c0-98d7-2bb9b25dba9a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:33.304053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a547a227-81da-472c-850b-1a368c53a54f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 12:21:33.304053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:33.304053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 12:21:33.304053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 12:21:33.304053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 12:21:33.304053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 12:21:33.304053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 12:21:33.304053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:33.304053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 12:21:33.304053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:33.304053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:21:33.304053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:33.304053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:33.304053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:33.304053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:33.304053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:33.304053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:33.305062 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:33.305062 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:33.305062 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:33.305062 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:33.305062 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:33.305062 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:33.305062 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 12:21:33.305062 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 12:21:33.305062 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 12:21:33.305062 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 12:21:33.305062 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:33.305062 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:33.305062 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:33.305062 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:33.305062 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:33.305062 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:33.306642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:33.306642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:33.306642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:33.306642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:33.306642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:33.306642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:33.306642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:33.306642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:33.306642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:33.306642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:33.306642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 12:21:33.306642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:33.306642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 12:21:33.306642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:33.310560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 454/923] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jun 29 12:21:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:33.312053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a547a227-81da-472c-850b-1a368c53a54f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:33.316624 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:33.427913 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a547a227-81da-472c-850b-1a368c53a54f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:33.439522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9c15d-980c-49c0-98d7-2bb9b25dba9a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:33.463159 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7e735-72f6-459e-8017-f38638f7d2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:33.463822 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7e735-72f6-459e-8017-f38638f7d2d3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:33.464109 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7e735-72f6-459e-8017-f38638f7d2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:33.464384 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7e735-72f6-459e-8017-f38638f7d2d3 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:33.465087 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7e735-72f6-459e-8017-f38638f7d2d3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:33.465391 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7e735-72f6-459e-8017-f38638f7d2d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:33.466774 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef7e735-72f6-459e-8017-f38638f7d2d3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:33.474451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 455/924] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jun 29 12:21:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:33.478201 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7e735-72f6-459e-8017-f38638f7d2d3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:33.490494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554fbcea-e47a-43c3-a022-bf61cf294114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:33.491892 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554fbcea-e47a-43c3-a022-bf61cf294114 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:33.492262 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554fbcea-e47a-43c3-a022-bf61cf294114 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:33.492608 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554fbcea-e47a-43c3-a022-bf61cf294114 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:33.507096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554fbcea-e47a-43c3-a022-bf61cf294114 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:21:33.520022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-554fbcea-e47a-43c3-a022-bf61cf294114 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 12:21:33.520022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:33.520022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:33.520022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:33.520022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:33.520022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:33.520022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:33.520022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:33.520022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:33.520022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:33.520022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:33.520022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:33.520022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:21:33.520022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:21:33.520022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:21:33.520022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:21:33.520965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:21:33.520965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:21:33.520965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:21:33.520965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:21:33.520965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:33.520965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:33.520965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:33.520965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:33.520965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:33.520965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:33.520965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:33.520965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:33.520965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:33.520965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:33.520965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:33.520965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:33.522105 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:33.522105 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:33.522105 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:33.522105 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:33.522105 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:21:33.522105 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:21:33.522105 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:21:33.522105 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:21:33.522105 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:21:33.522105 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:21:33.522105 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 12:21:33.522105 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:33.526725 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 470/925] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Jun 29 12:21:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:33.531328 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554fbcea-e47a-43c3-a022-bf61cf294114 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:33.548914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:33.562587 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:33.612642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7e735-72f6-459e-8017-f38638f7d2d3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:33.645467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554fbcea-e47a-43c3-a022-bf61cf294114 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:33.651507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c0611-bac5-4f16-be5a-bcd4fa9cd914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:33.652190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c0611-bac5-4f16-be5a-bcd4fa9cd914 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:33.652481 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c0611-bac5-4f16-be5a-bcd4fa9cd914 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:33.652765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c0611-bac5-4f16-be5a-bcd4fa9cd914 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:33.653484 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c0611-bac5-4f16-be5a-bcd4fa9cd914 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:33.654176 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c0611-bac5-4f16-be5a-bcd4fa9cd914 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:33.655325 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c0611-bac5-4f16-be5a-bcd4fa9cd914 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:33.666307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 456/926] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jun 29 12:21:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:33.667577 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c0611-bac5-4f16-be5a-bcd4fa9cd914 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:33.683908 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a44a5-9708-4b72-b89e-4520a5095375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:33.684964 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a44a5-9708-4b72-b89e-4520a5095375 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:33.685439 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a44a5-9708-4b72-b89e-4520a5095375 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:33.685890 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a44a5-9708-4b72-b89e-4520a5095375 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:33.686853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37a44a5-9708-4b72-b89e-4520a5095375 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:33.687336 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37a44a5-9708-4b72-b89e-4520a5095375 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:33.688878 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37a44a5-9708-4b72-b89e-4520a5095375 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:33.698987 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 471/927] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Jun 29 12:21:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:33.704342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a44a5-9708-4b72-b89e-4520a5095375 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:33.787559 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:33.795850 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c0611-bac5-4f16-be5a-bcd4fa9cd914 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:33.819430 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a44a5-9708-4b72-b89e-4520a5095375 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:33.829111 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:33.834819 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00329108-0b94-4bf3-bb59-45bb15668a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:33.836089 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00329108-0b94-4bf3-bb59-45bb15668a84 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:33.836347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00329108-0b94-4bf3-bb59-45bb15668a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:33.836940 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00329108-0b94-4bf3-bb59-45bb15668a84 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:33.838824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00329108-0b94-4bf3-bb59-45bb15668a84 None admin] RBAC: Adding query filter params (interface=public, service_id=35f308f8a3b0419d94225ee1599e568b, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:33.839243 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00329108-0b94-4bf3-bb59-45bb15668a84 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:33.840937 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00329108-0b94-4bf3-bb59-45bb15668a84 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:33.847207 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 457/928] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Jun 29 12:21:33 2022] GET /identity/v3/endpoints?service_id=35f308f8a3b0419d94225ee1599e568b&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:33.852162 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00329108-0b94-4bf3-bb59-45bb15668a84 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:33.859442 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520955a6-1e78-426d-ab4c-db47bff336b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:33.860415 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520955a6-1e78-426d-ab4c-db47bff336b0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:33.860858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520955a6-1e78-426d-ab4c-db47bff336b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:33.861295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520955a6-1e78-426d-ab4c-db47bff336b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:33.874138 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520955a6-1e78-426d-ab4c-db47bff336b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:33.876258 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520955a6-1e78-426d-ab4c-db47bff336b0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:33.894849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-520955a6-1e78-426d-ab4c-db47bff336b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:33.894849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:33.894849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:33.894849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:33.894849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:33.894849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:33.894849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:33.894849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:33.894849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:33.894849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:33.894849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:33.894849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:33.894849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:33.894849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:33.894849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:33.895801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:33.895801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:33.895801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:33.895801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:33.895801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:33.895801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:33.895801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:33.895801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:33.895801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:33.895801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:33.895801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:33.895801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:33.895801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:33.895801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:33.895801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:33.895801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:33.895801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:33.896945 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:33.896945 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:33.896945 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:33.896945 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:33.899421 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 472/929] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:33.914444 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520955a6-1e78-426d-ab4c-db47bff336b0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:33.974832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00329108-0b94-4bf3-bb59-45bb15668a84 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:34.011897 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a394f37c-0001-4241-a071-0ac21bb3df41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:34.012551 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394f37c-0001-4241-a071-0ac21bb3df41 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:34.012844 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394f37c-0001-4241-a071-0ac21bb3df41 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:34.013126 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394f37c-0001-4241-a071-0ac21bb3df41 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:34.013813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a394f37c-0001-4241-a071-0ac21bb3df41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:34.015212 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a394f37c-0001-4241-a071-0ac21bb3df41 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:34.021990 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 458/930] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Jun 29 12:21:33 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:21:34.036259 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520955a6-1e78-426d-ab4c-db47bff336b0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:34.066638 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:34.079799 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbb9d31-fbc2-424b-a32a-fc6f207a84f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:34.080486 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb9d31-fbc2-424b-a32a-fc6f207a84f3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:34.080773 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb9d31-fbc2-424b-a32a-fc6f207a84f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:34.081063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb9d31-fbc2-424b-a32a-fc6f207a84f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:34.081812 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbb9d31-fbc2-424b-a32a-fc6f207a84f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:34.082128 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbb9d31-fbc2-424b-a32a-fc6f207a84f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:34.083687 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbb9d31-fbc2-424b-a32a-fc6f207a84f3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:34.095828 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c7e54c-1f63-4a24-bb07-77015045a703 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:34.096173 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c7e54c-1f63-4a24-bb07-77015045a703 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:34.096397 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c7e54c-1f63-4a24-bb07-77015045a703 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:34.098055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 459/931] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:21:34.098861 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 473/932] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:34.103237 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673853a-88ae-4b88-8479-8ad81aeefd29 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:34.103749 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673853a-88ae-4b88-8479-8ad81aeefd29 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:34.104037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673853a-88ae-4b88-8479-8ad81aeefd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:34.104466 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbb9d31-fbc2-424b-a32a-fc6f207a84f3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:34.127597 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7673853a-88ae-4b88-8479-8ad81aeefd29 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:34.128985 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7673853a-88ae-4b88-8479-8ad81aeefd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:34.200412 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:34.215141 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 460/933] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 12:21:34.227678 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbb9d31-fbc2-424b-a32a-fc6f207a84f3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:34.239511 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673853a-88ae-4b88-8479-8ad81aeefd29 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:34.285412 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a287e-c313-4342-8fca-afeb41ad71f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:34.286236 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a287e-c313-4342-8fca-afeb41ad71f2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:34.286559 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a287e-c313-4342-8fca-afeb41ad71f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:34.286888 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a287e-c313-4342-8fca-afeb41ad71f2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:34.295264 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a287e-c313-4342-8fca-afeb41ad71f2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:34.296730 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a287e-c313-4342-8fca-afeb41ad71f2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:34.317075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6e4a287e-c313-4342-8fca-afeb41ad71f2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 12:21:34.317075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:34.317075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:34.317075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:34.317075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:34.317075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:34.317075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:34.317075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:34.317075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:34.317075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:34.317075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:34.317075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:34.317075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:21:34.317075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:21:34.317075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:21:34.317075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:21:34.318043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:34.318043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:34.318043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:21:34.318043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:34.318043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:21:34.318043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:34.318043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:34.318043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:34.318043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:34.318043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:34.318043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:34.318043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:34.318043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:34.318043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:34.318043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:34.318043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:34.318043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:34.318043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:34.319021 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:34.319021 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:34.319021 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:21:34.319021 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:21:34.319021 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:21:34.319021 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:21:34.319021 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:21:34.319021 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:21:34.319021 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 12:21:34.319021 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:34.323672 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 474/934] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jun 29 12:21:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:34.326020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a287e-c313-4342-8fca-afeb41ad71f2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:34.358678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673853a-88ae-4b88-8479-8ad81aeefd29 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:34.389539 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:34.396697 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b82f02-0754-4140-ae13-0d4ddd273071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:34.397850 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b82f02-0754-4140-ae13-0d4ddd273071 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:34.398340 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b82f02-0754-4140-ae13-0d4ddd273071 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:34.398950 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b82f02-0754-4140-ae13-0d4ddd273071 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:34.399841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b82f02-0754-4140-ae13-0d4ddd273071 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:34.401424 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b82f02-0754-4140-ae13-0d4ddd273071 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:34.414262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b5b82f02-0754-4140-ae13-0d4ddd273071 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 12:21:34.414262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:34.414262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 12:21:34.414262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 12:21:34.414262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 12:21:34.414262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 12:21:34.414262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 12:21:34.414262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:34.414262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 12:21:34.414262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:34.414262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:21:34.414262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:34.414262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:34.414262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:34.414262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:34.414262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:34.415891 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:34.415891 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:34.415891 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:34.415891 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:34.415891 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:34.415891 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:34.415891 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:34.415891 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 12:21:34.415891 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 12:21:34.415891 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 12:21:34.415891 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 12:21:34.415891 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:34.415891 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:34.415891 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:34.415891 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:34.415891 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:34.416755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:34.416755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:34.416755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:34.416755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:34.416755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:34.416755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:34.416755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:34.416755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:34.416755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:34.416755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:34.416755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:34.416755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 12:21:34.416755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:34.416755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 29 12:21:34.416755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:34.418853 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 461/935] 10.222.0.52 () {56 vars in 1132 bytes} [Wed Jun 29 12:21:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:34.425144 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b82f02-0754-4140-ae13-0d4ddd273071 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:34.452978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a287e-c313-4342-8fca-afeb41ad71f2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:34.483503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:34.493691 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7c702-bf57-44f5-88e9-2050ed053867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:34.494372 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7c702-bf57-44f5-88e9-2050ed053867 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:34.494706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7c702-bf57-44f5-88e9-2050ed053867 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:34.494991 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7c702-bf57-44f5-88e9-2050ed053867 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:34.495934 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7c702-bf57-44f5-88e9-2050ed053867 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:34.496434 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7c702-bf57-44f5-88e9-2050ed053867 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:34.498179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff7c702-bf57-44f5-88e9-2050ed053867 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:34.522743 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 475/936] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Jun 29 12:21:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:34.524270 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7c702-bf57-44f5-88e9-2050ed053867 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:34.532497 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b82f02-0754-4140-ae13-0d4ddd273071 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:34.578534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259818c8-8b9c-41e1-8d8b-6f93b82288c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:34.579040 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259818c8-8b9c-41e1-8d8b-6f93b82288c7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:34.579504 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259818c8-8b9c-41e1-8d8b-6f93b82288c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:34.580028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259818c8-8b9c-41e1-8d8b-6f93b82288c7 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:34.580968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259818c8-8b9c-41e1-8d8b-6f93b82288c7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:34.581610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259818c8-8b9c-41e1-8d8b-6f93b82288c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:34.583204 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259818c8-8b9c-41e1-8d8b-6f93b82288c7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:34.590496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 462/937] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jun 29 12:21:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:34.595872 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259818c8-8b9c-41e1-8d8b-6f93b82288c7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:34.645094 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff7c702-bf57-44f5-88e9-2050ed053867 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:34.647941 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:34.686689 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c5cb9-72f0-43d0-8c2a-63d0c7d0fe41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:34.687761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c5cb9-72f0-43d0-8c2a-63d0c7d0fe41 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:34.688216 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c5cb9-72f0-43d0-8c2a-63d0c7d0fe41 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:34.689050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c5cb9-72f0-43d0-8c2a-63d0c7d0fe41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:34.702830 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c5cb9-72f0-43d0-8c2a-63d0c7d0fe41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:34.704740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c5cb9-72f0-43d0-8c2a-63d0c7d0fe41 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:34.715348 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259818c8-8b9c-41e1-8d8b-6f93b82288c7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:34.728077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-cf2c5cb9-72f0-43d0-8c2a-63d0c7d0fe41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:34.728077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:34.728077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:34.728077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:34.728077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:34.728077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:34.728077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:34.728077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:34.728077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:34.728077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:34.728077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:34.728077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:34.728077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:34.728077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:34.728077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:34.729322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:34.729322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:34.729322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:34.729322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:34.729322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:34.729322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:34.729322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:34.729322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:34.729322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:34.729322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:34.729322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:34.729322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:34.729322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:34.729322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:34.729322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:34.729322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:34.729322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:34.730398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:34.730398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:34.730398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:34.730398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:34.730398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 476/938] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:34.730728 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85addc2a-0064-40d7-a2ac-c1963efb2ec1 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:34.730813 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85addc2a-0064-40d7-a2ac-c1963efb2ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:34.731093 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85addc2a-0064-40d7-a2ac-c1963efb2ec1 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:34.732579 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 477/939] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:34.735381 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85addc2a-0064-40d7-a2ac-c1963efb2ec1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:34.741388 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:34.761601 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858baab4-e80e-45db-bda9-b34c05c127d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:34.762561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858baab4-e80e-45db-bda9-b34c05c127d5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:34.763011 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858baab4-e80e-45db-bda9-b34c05c127d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:34.763287 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858baab4-e80e-45db-bda9-b34c05c127d5 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:34.764146 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858baab4-e80e-45db-bda9-b34c05c127d5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:34.764635 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858baab4-e80e-45db-bda9-b34c05c127d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:34.766155 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858baab4-e80e-45db-bda9-b34c05c127d5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:34.771971 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 463/940] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jun 29 12:21:34 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:34.773513 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1de3a4-4243-4b03-9448-b9b0aee9c293 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:34.774008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1de3a4-4243-4b03-9448-b9b0aee9c293 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:34.774550 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1de3a4-4243-4b03-9448-b9b0aee9c293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:34.793473 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8b1de3a4-4243-4b03-9448-b9b0aee9c293 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:34.794701 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8b1de3a4-4243-4b03-9448-b9b0aee9c293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:34.850222 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85addc2a-0064-40d7-a2ac-c1963efb2ec1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:34.867365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 464/941] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:34.875300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1de3a4-4243-4b03-9448-b9b0aee9c293 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:34.888403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064d451-d849-407f-8d65-3980a9f2d02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:34.889282 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064d451-d849-407f-8d65-3980a9f2d02a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:34.889729 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064d451-d849-407f-8d65-3980a9f2d02a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:34.890134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064d451-d849-407f-8d65-3980a9f2d02a None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:34.891090 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064d451-d849-407f-8d65-3980a9f2d02a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:34.891398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064d451-d849-407f-8d65-3980a9f2d02a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:34.892759 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064d451-d849-407f-8d65-3980a9f2d02a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:34.905635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 478/942] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:34.906924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064d451-d849-407f-8d65-3980a9f2d02a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:34.927404 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:34.983957 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1de3a4-4243-4b03-9448-b9b0aee9c293 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:34.997701 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:35.022129 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a17b4-fbcf-40eb-b701-6106cad30df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:35.022911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a17b4-fbcf-40eb-b701-6106cad30df9 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:35.023211 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a17b4-fbcf-40eb-b701-6106cad30df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:35.023493 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a17b4-fbcf-40eb-b701-6106cad30df9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:35.024267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a17b4-fbcf-40eb-b701-6106cad30df9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:35.025677 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a17b4-fbcf-40eb-b701-6106cad30df9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:35.027319 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064d451-d849-407f-8d65-3980a9f2d02a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:35.046497 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 465/943] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Jun 29 12:21:34 2022] POST /identity/v3/endpoints => generated 347 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:21:35.047137 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a17b4-fbcf-40eb-b701-6106cad30df9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:35.091333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c06f22-a5cb-47bd-8b2f-c72ecabda799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:35.092708 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c06f22-a5cb-47bd-8b2f-c72ecabda799 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:35.093180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c06f22-a5cb-47bd-8b2f-c72ecabda799 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:35.093638 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c06f22-a5cb-47bd-8b2f-c72ecabda799 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:35.094638 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c06f22-a5cb-47bd-8b2f-c72ecabda799 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:35.097158 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c06f22-a5cb-47bd-8b2f-c72ecabda799 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:35.099939 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: WARNING keystone.server.flask.common [None req-09c06f22-a5cb-47bd-8b2f-c72ecabda799 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 29 12:21:35.128068 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.sql.core [None req-09c06f22-a5cb-47bd-8b2f-c72ecabda799 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 29 12:21:35.128068 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 29 12:21:35.128068 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [parameters: {'id': '65e00edc3f8d4ea3beb9297efc78415f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 29 12:21:35.128068 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80169) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 29 12:21:35.152287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-09c06f22-a5cb-47bd-8b2f-c72ecabda799 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 65e00edc3f8d4ea3beb9297efc78415f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 65e00edc3f8d4ea3beb9297efc78415f. Jun 29 12:21:35.152287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:35.152287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 12:21:35.152287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 12:21:35.152287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 12:21:35.152287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 12:21:35.152287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 12:21:35.152287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 12:21:35.152287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 12:21:35.152287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application conn.query(q) Jun 29 12:21:35.152287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 12:21:35.152287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 12:21:35.152287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 12:21:35.153180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result.read() Jun 29 12:21:35.153180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 12:21:35.153180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 12:21:35.153180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 12:21:35.153180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 12:21:35.153180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 12:21:35.153180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 12:21:35.153180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 12:21:35.153180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 12:21:35.153180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 29 12:21:35.153180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:35.153180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 29 12:21:35.153180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:35.153180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:35.153180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 29 12:21:35.153180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 29 12:21:35.153180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 29 12:21:35.153180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 29 12:21:35.154138 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 12:21:35.154138 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application next(self.gen) Jun 29 12:21:35.154138 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 29 12:21:35.154138 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application yield resource Jun 29 12:21:35.154138 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 12:21:35.154138 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application next(self.gen) Jun 29 12:21:35.154138 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 29 12:21:35.154138 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.session.rollback() Jun 29 12:21:35.154138 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 29 12:21:35.154138 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.force_reraise() Jun 29 12:21:35.154138 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 29 12:21:35.154138 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise self.value Jun 29 12:21:35.154138 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 29 12:21:35.154138 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 29 12:21:35.154138 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 29 12:21:35.154138 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application session.commit() Jun 29 12:21:35.154138 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 29 12:21:35.155101 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 29 12:21:35.155101 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 29 12:21:35.155101 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._prepare_impl() Jun 29 12:21:35.155101 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 29 12:21:35.155101 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.session.flush() Jun 29 12:21:35.155101 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 29 12:21:35.155101 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._flush(objects) Jun 29 12:21:35.155101 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 29 12:21:35.155101 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 29 12:21:35.155101 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 29 12:21:35.155101 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application compat.raise_( Jun 29 12:21:35.155101 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 12:21:35.155101 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception Jun 29 12:21:35.155101 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 29 12:21:35.155101 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application flush_context.execute() Jun 29 12:21:35.155101 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 29 12:21:35.155101 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rec.execute(self) Jun 29 12:21:35.159746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 29 12:21:35.159746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 29 12:21:35.159746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 29 12:21:35.159746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 29 12:21:35.159746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 29 12:21:35.159746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 29 12:21:35.159746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 29 12:21:35.159746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 29 12:21:35.159746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 29 12:21:35.159746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 29 12:21:35.159746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 29 12:21:35.159746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 29 12:21:35.159746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 29 12:21:35.159746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 29 12:21:35.159746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 29 12:21:35.160626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 29 12:21:35.160626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 12:21:35.160626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception Jun 29 12:21:35.160626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 12:21:35.160626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 12:21:35.160626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 12:21:35.160626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 12:21:35.160626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 12:21:35.160626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 12:21:35.160626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 12:21:35.160626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application conn.query(q) Jun 29 12:21:35.160626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 12:21:35.160626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 12:21:35.160626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 12:21:35.160626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result.read() Jun 29 12:21:35.160626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 12:21:35.160626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 12:21:35.161531 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 12:21:35.161531 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 12:21:35.161531 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 12:21:35.161531 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 12:21:35.161531 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 12:21:35.161531 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 12:21:35.161531 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 29 12:21:35.161531 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 29 12:21:35.161531 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [parameters: {'id': '65e00edc3f8d4ea3beb9297efc78415f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 29 12:21:35.161531 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 29 12:21:35.161531 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:35.161531 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:21:35.161531 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:35.161531 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:35.161531 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 29 12:21:35.162331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 29 12:21:35.162331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 29 12:21:35.162331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 29 12:21:35.162331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 29 12:21:35.162331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:35.162331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:21:35.162331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:35.162331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:35.162331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:35.162331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:35.162331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:35.162331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:35.162331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:35.162331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:35.162331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:35.162331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:35.162331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:35.163312 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:35.163312 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 29 12:21:35.163312 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 29 12:21:35.163312 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:35.163312 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:35.163312 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 29 12:21:35.163312 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 29 12:21:35.163312 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 65e00edc3f8d4ea3beb9297efc78415f. Jun 29 12:21:35.163312 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:35.163312 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 479/944] 10.222.0.52 () {60 vars in 1140 bytes} [Wed Jun 29 12:21:34 2022] POST /identity/v3/projects => generated 262 bytes in 250 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:35.166144 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c06f22-a5cb-47bd-8b2f-c72ecabda799 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:35.191505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a17b4-fbcf-40eb-b701-6106cad30df9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:35.239531 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677a249-24e0-41ab-868c-054afa1ca351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:35.240384 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677a249-24e0-41ab-868c-054afa1ca351 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:35.240675 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677a249-24e0-41ab-868c-054afa1ca351 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:35.240967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677a249-24e0-41ab-868c-054afa1ca351 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:35.245179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:35.252748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9677a249-24e0-41ab-868c-054afa1ca351 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:35.255092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9677a249-24e0-41ab-868c-054afa1ca351 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:35.271920 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9677a249-24e0-41ab-868c-054afa1ca351 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:35.271920 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:35.271920 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:35.271920 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:35.271920 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:35.271920 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:35.271920 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:35.271920 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:35.271920 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:35.271920 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:35.271920 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:35.271920 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:35.271920 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:21:35.271920 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:21:35.271920 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:21:35.272939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:21:35.272939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:35.272939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:35.272939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:35.272939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:35.272939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:35.272939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:35.272939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:35.272939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:35.272939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:35.272939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:35.272939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:35.272939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:35.272939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:35.272939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:35.272939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:35.272939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:35.273843 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:21:35.273843 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:21:35.273843 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:21:35.273843 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:21:35.273843 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:21:35.273843 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:21:35.273843 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:35.273843 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:35.273843 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 466/945] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:21:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:35.281168 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677a249-24e0-41ab-868c-054afa1ca351 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:35.296600 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c06f22-a5cb-47bd-8b2f-c72ecabda799 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:35.315304 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:35.339186 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e03e70a-3577-4b41-819c-7a34944ccc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:35.340053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03e70a-3577-4b41-819c-7a34944ccc16 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:35.340414 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03e70a-3577-4b41-819c-7a34944ccc16 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:35.340766 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03e70a-3577-4b41-819c-7a34944ccc16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:35.354868 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e03e70a-3577-4b41-819c-7a34944ccc16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:35.356751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e03e70a-3577-4b41-819c-7a34944ccc16 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:35.378626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7e03e70a-3577-4b41-819c-7a34944ccc16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:35.378626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:35.378626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:35.378626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:35.378626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:35.378626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:35.378626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:35.378626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:35.378626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:35.378626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:35.378626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:35.378626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:35.378626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:21:35.378626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:21:35.378626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:21:35.379760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:21:35.379760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:35.379760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:35.379760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:35.379760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:35.379760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:35.379760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:35.379760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:35.379760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:35.379760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:35.379760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:35.379760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:35.379760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:35.379760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:35.379760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:35.379760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:35.379760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:35.380869 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:21:35.380869 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:21:35.380869 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:21:35.380869 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:21:35.380869 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:21:35.380869 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:21:35.380869 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:35.380869 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:35.386699 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 480/946] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:21:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:35.388720 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e03e70a-3577-4b41-819c-7a34944ccc16 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:35.408546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677a249-24e0-41ab-868c-054afa1ca351 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:35.454022 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bebb43-d458-4f44-9bff-d01e52fe4250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:35.454992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bebb43-d458-4f44-9bff-d01e52fe4250 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:35.455544 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bebb43-d458-4f44-9bff-d01e52fe4250 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:35.455967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bebb43-d458-4f44-9bff-d01e52fe4250 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:35.457438 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bebb43-d458-4f44-9bff-d01e52fe4250 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:35.457904 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bebb43-d458-4f44-9bff-d01e52fe4250 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:35.459505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bebb43-d458-4f44-9bff-d01e52fe4250 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:35.480375 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 467/947] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:21:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:35.482023 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bebb43-d458-4f44-9bff-d01e52fe4250 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:35.504662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e03e70a-3577-4b41-819c-7a34944ccc16 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:35.532999 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:35.545496 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd105b-b4bf-4d1b-bca6-7d98c858665c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:35.546738 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd105b-b4bf-4d1b-bca6-7d98c858665c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:35.547103 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd105b-b4bf-4d1b-bca6-7d98c858665c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:35.547459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd105b-b4bf-4d1b-bca6-7d98c858665c None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:35.548292 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd105b-b4bf-4d1b-bca6-7d98c858665c None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:35.548676 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd105b-b4bf-4d1b-bca6-7d98c858665c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:35.550312 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd105b-b4bf-4d1b-bca6-7d98c858665c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:35.576309 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 481/948] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jun 29 12:21:35 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:35.596319 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:35.605058 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bebb43-d458-4f44-9bff-d01e52fe4250 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:35.644044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780bef96-c856-4e90-8648-48a1a3b6fdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:35.644890 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bef96-c856-4e90-8648-48a1a3b6fdaf None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:35.645182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bef96-c856-4e90-8648-48a1a3b6fdaf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:35.645463 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bef96-c856-4e90-8648-48a1a3b6fdaf None admin] PATH_INFO: `/v3/projects/f043ade9e4f748d4adc010ffb0a47a46/users/8c10852a728844c6ae0a48e3772d95b7/roles/7dedcc77e86a4fee94bf84105c0ffd71` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:35.648148 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780bef96-c856-4e90-8648-48a1a3b6fdaf None admin] RBAC: Authorizing `identity:create_grant(project_id=f043ade9e4f748d4adc010ffb0a47a46, user_id=8c10852a728844c6ae0a48e3772d95b7, role_id=7dedcc77e86a4fee94bf84105c0ffd71)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:35.650568 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780bef96-c856-4e90-8648-48a1a3b6fdaf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:35.668760 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 468/949] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 12:21:35 2022] PUT /identity/v3/projects/f043ade9e4f748d4adc010ffb0a47a46/users/8c10852a728844c6ae0a48e3772d95b7/roles/7dedcc77e86a4fee94bf84105c0ffd71 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:21:36.222238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab86add-4a90-4845-9f23-d4196fb6edb9 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:36.222815 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab86add-4a90-4845-9f23-d4196fb6edb9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:36.222815 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab86add-4a90-4845-9f23-d4196fb6edb9 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:36.224472 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 482/950] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:36.230820 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74149163-03df-46bb-a9fb-25299b8d4dbb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:36.231292 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74149163-03df-46bb-a9fb-25299b8d4dbb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:36.231372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74149163-03df-46bb-a9fb-25299b8d4dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:36.249779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-74149163-03df-46bb-a9fb-25299b8d4dbb None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:36.251036 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-74149163-03df-46bb-a9fb-25299b8d4dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:36.321698 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 469/951] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:36.345354 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab86add-4a90-4845-9f23-d4196fb6edb9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:36.460249 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab86add-4a90-4845-9f23-d4196fb6edb9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:36.496594 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc53727-2eaf-49f0-9e3f-c052e9c2b3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:36.497601 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc53727-2eaf-49f0-9e3f-c052e9c2b3ec None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:36.498044 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc53727-2eaf-49f0-9e3f-c052e9c2b3ec None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:36.498489 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc53727-2eaf-49f0-9e3f-c052e9c2b3ec None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:36.499279 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc53727-2eaf-49f0-9e3f-c052e9c2b3ec None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:36.500883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc53727-2eaf-49f0-9e3f-c052e9c2b3ec None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:36.513427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7dc53727-2eaf-49f0-9e3f-c052e9c2b3ec None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 12:21:36.513427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:36.513427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 12:21:36.513427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 12:21:36.513427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 12:21:36.513427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 12:21:36.513427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 12:21:36.513427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:36.513427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 12:21:36.513427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:36.513427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:21:36.513427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:36.513427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:36.513427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:36.513427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:36.513427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:36.513427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:36.514679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:36.514679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:36.514679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:36.514679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:36.514679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:36.514679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:36.514679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 12:21:36.514679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 12:21:36.514679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 12:21:36.514679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 12:21:36.514679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:36.514679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:36.514679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:36.514679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:36.514679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:36.514679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:36.515715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:36.515715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:36.515715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:36.515715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:36.515715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:36.515715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:36.515715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:36.515715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:36.515715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:36.515715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:36.515715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 12:21:36.515715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:36.515715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 12:21:36.515715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:36.518328 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 483/952] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jun 29 12:21:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:36.525710 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74149163-03df-46bb-a9fb-25299b8d4dbb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:36.605959 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:36.628942 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74149163-03df-46bb-a9fb-25299b8d4dbb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:36.664575 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efa3936-abc3-4d4c-b3b6-e1bc2dde5e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:36.666337 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa3936-abc3-4d4c-b3b6-e1bc2dde5e9c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:36.666667 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa3936-abc3-4d4c-b3b6-e1bc2dde5e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:36.666948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efa3936-abc3-4d4c-b3b6-e1bc2dde5e9c None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:36.667659 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efa3936-abc3-4d4c-b3b6-e1bc2dde5e9c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:36.667966 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efa3936-abc3-4d4c-b3b6-e1bc2dde5e9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:36.669320 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efa3936-abc3-4d4c-b3b6-e1bc2dde5e9c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:36.674647 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 470/953] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jun 29 12:21:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:36.680287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc53727-2eaf-49f0-9e3f-c052e9c2b3ec None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:36.733407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:36.783875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc53727-2eaf-49f0-9e3f-c052e9c2b3ec None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:36.819146 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e359005-4beb-47da-8903-ab3be1d8aeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:36.819819 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e359005-4beb-47da-8903-ab3be1d8aeb5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:36.820102 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e359005-4beb-47da-8903-ab3be1d8aeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:36.820379 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e359005-4beb-47da-8903-ab3be1d8aeb5 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:36.821090 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e359005-4beb-47da-8903-ab3be1d8aeb5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:36.821489 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e359005-4beb-47da-8903-ab3be1d8aeb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:36.822907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e359005-4beb-47da-8903-ab3be1d8aeb5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:36.828749 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 484/954] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jun 29 12:21:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:36.834292 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efa3936-abc3-4d4c-b3b6-e1bc2dde5e9c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:36.949821 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efa3936-abc3-4d4c-b3b6-e1bc2dde5e9c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:36.984164 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ba173-0e62-425f-8290-4a18d0cd5d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:36.984832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ba173-0e62-425f-8290-4a18d0cd5d19 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:36.985117 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ba173-0e62-425f-8290-4a18d0cd5d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:36.985396 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ba173-0e62-425f-8290-4a18d0cd5d19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:36.986129 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6ba173-0e62-425f-8290-4a18d0cd5d19 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:36.987519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6ba173-0e62-425f-8290-4a18d0cd5d19 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:37.002178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:37.008926 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 471/955] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Jun 29 12:21:36 2022] POST /identity/v3/endpoints => generated 349 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:21:37.198868 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e04dbb-89c9-4d95-848f-7acd03616a15 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:37.199085 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e04dbb-89c9-4d95-848f-7acd03616a15 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:37.199371 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e04dbb-89c9-4d95-848f-7acd03616a15 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:37.201055 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 485/956] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:37.206180 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915e3c0-88c0-44de-9227-3fdb84f193b9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:37.206751 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915e3c0-88c0-44de-9227-3fdb84f193b9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:37.207250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915e3c0-88c0-44de-9227-3fdb84f193b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:37.224442 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f915e3c0-88c0-44de-9227-3fdb84f193b9 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:37.226024 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f915e3c0-88c0-44de-9227-3fdb84f193b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:37.309310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 472/957] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:37.332046 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e04dbb-89c9-4d95-848f-7acd03616a15 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:37.440167 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e04dbb-89c9-4d95-848f-7acd03616a15 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:37.454898 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:37.475456 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b7290-e562-441b-84f3-35ed2c8ec155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:37.476257 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b7290-e562-441b-84f3-35ed2c8ec155 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:37.476537 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b7290-e562-441b-84f3-35ed2c8ec155 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:37.476811 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b7290-e562-441b-84f3-35ed2c8ec155 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:37.477456 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b7290-e562-441b-84f3-35ed2c8ec155 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:37.479417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b7290-e562-441b-84f3-35ed2c8ec155 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:37.491624 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e81b7290-e562-441b-84f3-35ed2c8ec155 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 12:21:37.491624 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:37.491624 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 12:21:37.491624 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 12:21:37.491624 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 12:21:37.491624 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 12:21:37.491624 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 12:21:37.491624 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:37.491624 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 12:21:37.491624 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:37.491624 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:21:37.491624 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:37.491624 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:37.491624 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:37.491624 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:37.491624 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:37.491624 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:37.492777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:37.492777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:37.492777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:37.492777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:37.492777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:37.492777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:37.492777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 12:21:37.492777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 12:21:37.492777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 12:21:37.492777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 12:21:37.492777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:37.492777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:37.492777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:37.492777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:37.492777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:37.492777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:37.495130 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:37.495130 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:37.495130 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:37.495130 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:37.495130 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:37.495130 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:37.495130 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:37.495130 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:37.495130 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:37.495130 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:37.495130 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 12:21:37.495130 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:37.495130 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 12:21:37.495130 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:37.495130 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 486/958] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:21:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:37.503557 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915e3c0-88c0-44de-9227-3fdb84f193b9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:37.608448 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915e3c0-88c0-44de-9227-3fdb84f193b9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:37.633245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:37.645647 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3989d47-3977-4d82-a1fd-3b830b385bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:37.646317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3989d47-3977-4d82-a1fd-3b830b385bfb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:37.646925 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3989d47-3977-4d82-a1fd-3b830b385bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:37.647208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3989d47-3977-4d82-a1fd-3b830b385bfb None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:37.647923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3989d47-3977-4d82-a1fd-3b830b385bfb None admin] RBAC: Adding query filter params (name=compute) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:37.648228 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3989d47-3977-4d82-a1fd-3b830b385bfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:37.649823 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3989d47-3977-4d82-a1fd-3b830b385bfb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:37.655066 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 473/959] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jun 29 12:21:37 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:37.660722 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b7290-e562-441b-84f3-35ed2c8ec155 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:37.763252 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b7290-e562-441b-84f3-35ed2c8ec155 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:37.800333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f650299-f8c2-4e63-b1ec-71596629ac31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:37.801000 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f650299-f8c2-4e63-b1ec-71596629ac31 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:37.801283 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f650299-f8c2-4e63-b1ec-71596629ac31 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:37.801562 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f650299-f8c2-4e63-b1ec-71596629ac31 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:37.802271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f650299-f8c2-4e63-b1ec-71596629ac31 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:37.802603 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f650299-f8c2-4e63-b1ec-71596629ac31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:37.803971 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f650299-f8c2-4e63-b1ec-71596629ac31 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:37.809560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 487/960] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jun 29 12:21:37 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:37.812673 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018eb30-d7cc-40bf-9c91-44b7694e5a59 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:37.813101 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018eb30-d7cc-40bf-9c91-44b7694e5a59 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:37.813387 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018eb30-d7cc-40bf-9c91-44b7694e5a59 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:37.819654 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 474/961] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:37.828378 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a949e-ad1b-484c-95a6-3bb168258b04 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:37.828717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a949e-ad1b-484c-95a6-3bb168258b04 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:37.828947 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a949e-ad1b-484c-95a6-3bb168258b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:37.847511 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2b8a949e-ad1b-484c-95a6-3bb168258b04 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:37.849067 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2b8a949e-ad1b-484c-95a6-3bb168258b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:37.891622 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:37.909377 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719b1b3-e979-4628-8cc1-23146e597b37 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:37.910268 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719b1b3-e979-4628-8cc1-23146e597b37 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:37.910803 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719b1b3-e979-4628-8cc1-23146e597b37 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:37.912991 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 475/962] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:21:37.918611 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b86d8-014e-4747-aa94-db423be8df0a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:37.918922 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b86d8-014e-4747-aa94-db423be8df0a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:37.919171 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b86d8-014e-4747-aa94-db423be8df0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:37.938166 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 488/963] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:37.938598 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-af7b86d8-014e-4747-aa94-db423be8df0a None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:37.939835 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-af7b86d8-014e-4747-aa94-db423be8df0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:37.970616 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a949e-ad1b-484c-95a6-3bb168258b04 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:38.021784 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 476/964] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 12:21:38.047039 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b86d8-014e-4747-aa94-db423be8df0a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:38.090093 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a949e-ad1b-484c-95a6-3bb168258b04 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:38.128306 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768fd314-bf09-4386-90e1-7b0d1d73b06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:38.129448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fd314-bf09-4386-90e1-7b0d1d73b06c None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:38.129950 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fd314-bf09-4386-90e1-7b0d1d73b06c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:38.130465 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fd314-bf09-4386-90e1-7b0d1d73b06c None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:38.131501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768fd314-bf09-4386-90e1-7b0d1d73b06c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:38.133237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768fd314-bf09-4386-90e1-7b0d1d73b06c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:38.148141 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.sql.core [None req-768fd314-bf09-4386-90e1-7b0d1d73b06c None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 29 12:21:38.148141 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 29 12:21:38.148141 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [parameters: {'user_id': 'f149abe8e4e84200a3aebafd36773d28', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 29 12:21:38.148141 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80169) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 29 12:21:38.155707 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-768fd314-bf09-4386-90e1-7b0d1d73b06c None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 29 12:21:38.155707 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:38.155707 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 12:21:38.155707 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 12:21:38.155707 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 12:21:38.155707 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 12:21:38.155707 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 12:21:38.155707 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 12:21:38.155707 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 12:21:38.155707 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application conn.query(q) Jun 29 12:21:38.155707 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 12:21:38.155707 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 12:21:38.155707 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 12:21:38.155707 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result.read() Jun 29 12:21:38.155707 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 12:21:38.156685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 12:21:38.156685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 12:21:38.156685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 12:21:38.156685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 12:21:38.156685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 12:21:38.156685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 12:21:38.156685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 12:21:38.156685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 29 12:21:38.156685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:38.156685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 29 12:21:38.156685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:38.156685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:38.156685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 29 12:21:38.156685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 29 12:21:38.156685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 29 12:21:38.156685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 29 12:21:38.156685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 12:21:38.156685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application next(self.gen) Jun 29 12:21:38.158340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 29 12:21:38.158340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application yield resource Jun 29 12:21:38.158340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 29 12:21:38.158340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application next(self.gen) Jun 29 12:21:38.158340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 29 12:21:38.158340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.session.rollback() Jun 29 12:21:38.158340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 29 12:21:38.158340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.force_reraise() Jun 29 12:21:38.158340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 29 12:21:38.158340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise self.value Jun 29 12:21:38.158340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 29 12:21:38.158340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 29 12:21:38.158340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 29 12:21:38.158340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application session.commit() Jun 29 12:21:38.158340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 29 12:21:38.158340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 29 12:21:38.158340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 29 12:21:38.159307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._prepare_impl() Jun 29 12:21:38.159307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 29 12:21:38.159307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.session.flush() Jun 29 12:21:38.159307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 29 12:21:38.159307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._flush(objects) Jun 29 12:21:38.159307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 29 12:21:38.159307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 29 12:21:38.159307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 29 12:21:38.159307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application compat.raise_( Jun 29 12:21:38.159307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 12:21:38.159307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception Jun 29 12:21:38.159307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 29 12:21:38.159307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application flush_context.execute() Jun 29 12:21:38.159307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 29 12:21:38.159307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rec.execute(self) Jun 29 12:21:38.159307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 29 12:21:38.159307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 29 12:21:38.161271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 29 12:21:38.161271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 29 12:21:38.161271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 29 12:21:38.161271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 29 12:21:38.161271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 29 12:21:38.161271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 29 12:21:38.161271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 29 12:21:38.161271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 29 12:21:38.161271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 29 12:21:38.161271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 29 12:21:38.161271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 29 12:21:38.161271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 29 12:21:38.161271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 29 12:21:38.161271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 29 12:21:38.161271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 29 12:21:38.161271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception Jun 29 12:21:38.163567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 29 12:21:38.163567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 29 12:21:38.163567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 29 12:21:38.163567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 29 12:21:38.163567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 29 12:21:38.163567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result = self._query(query) Jun 29 12:21:38.163567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 29 12:21:38.163567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application conn.query(q) Jun 29 12:21:38.163567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 29 12:21:38.163567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 29 12:21:38.163567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 29 12:21:38.163567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result.read() Jun 29 12:21:38.163567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 29 12:21:38.163567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 29 12:21:38.163567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 29 12:21:38.163567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 29 12:21:38.166305 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 29 12:21:38.166305 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 29 12:21:38.166305 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 29 12:21:38.166305 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 29 12:21:38.166305 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 29 12:21:38.166305 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 29 12:21:38.166305 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [parameters: {'user_id': 'f149abe8e4e84200a3aebafd36773d28', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 29 12:21:38.166305 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 29 12:21:38.166305 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:38.166305 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:21:38.166305 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:38.166305 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:38.166305 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:38.166305 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:38.166305 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:38.169522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:38.169522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:38.169522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:38.169522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:38.169522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:38.169522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:38.169522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:38.169522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 29 12:21:38.169522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 29 12:21:38.169522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:38.169522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:38.169522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:21:38.169522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:38.169522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:21:38.169522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:38.169522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 29 12:21:38.169522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 29 12:21:38.171827 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 29 12:21:38.171827 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 29 12:21:38.171827 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 29 12:21:38.171827 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 29 12:21:38.171827 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 29 12:21:38.171827 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:38.171827 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 489/965] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Jun 29 12:21:37 2022] POST /identity/v3/users => generated 160 bytes in 187 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:38.171827 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768fd314-bf09-4386-90e1-7b0d1d73b06c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:38.195216 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b86d8-014e-4747-aa94-db423be8df0a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:38.205378 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:38.231359 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:38.265757 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e320429-fdaa-41f7-b236-1da02eaa6ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:38.267175 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e320429-fdaa-41f7-b236-1da02eaa6ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:38.267608 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e320429-fdaa-41f7-b236-1da02eaa6ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:38.268077 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e320429-fdaa-41f7-b236-1da02eaa6ea5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:38.284653 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e320429-fdaa-41f7-b236-1da02eaa6ea5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:21:38.289264 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768fd314-bf09-4386-90e1-7b0d1d73b06c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:38.296798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0e320429-fdaa-41f7-b236-1da02eaa6ea5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 12:21:38.296798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:38.296798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:38.296798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:38.296798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:38.296798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:38.296798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:38.296798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:38.296798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:38.296798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:38.296798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:38.296798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:38.296798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:21:38.296798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:21:38.296798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:21:38.296798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:21:38.297835 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:21:38.297835 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:21:38.297835 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:21:38.297835 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:21:38.297835 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:38.297835 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:38.297835 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:38.297835 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:38.297835 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:38.297835 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:38.297835 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:38.297835 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:38.297835 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:38.297835 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:38.297835 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:38.297835 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:38.299661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:38.299661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:38.299661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:38.299661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:38.299661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:21:38.299661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:21:38.299661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:21:38.299661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:21:38.299661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:21:38.299661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:21:38.299661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 12:21:38.299661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:38.302841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 477/966] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Jun 29 12:21:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:38.307965 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e320429-fdaa-41f7-b236-1da02eaa6ea5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:38.334600 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b736c2d-778f-469a-8a56-fd3b119b2fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:38.335419 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b736c2d-778f-469a-8a56-fd3b119b2fdc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:38.335701 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b736c2d-778f-469a-8a56-fd3b119b2fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:38.335974 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b736c2d-778f-469a-8a56-fd3b119b2fdc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:38.344522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b736c2d-778f-469a-8a56-fd3b119b2fdc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:38.346225 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b736c2d-778f-469a-8a56-fd3b119b2fdc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:38.362835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9b736c2d-778f-469a-8a56-fd3b119b2fdc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 12:21:38.362835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:38.362835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:38.362835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:38.362835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:38.362835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:38.362835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:38.362835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:38.362835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:38.362835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:38.362835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:38.362835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:38.362835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:21:38.362835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:21:38.362835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:21:38.362835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:21:38.363962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:38.363962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:38.363962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:21:38.363962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:38.363962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:21:38.363962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:38.363962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:38.363962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:38.363962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:38.363962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:38.363962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:38.363962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:38.363962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:38.363962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:38.363962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:38.363962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:38.363962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:38.363962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:38.364904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:38.364904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:38.364904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:21:38.364904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:21:38.364904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:21:38.364904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:21:38.364904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:21:38.364904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:21:38.364904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 29 12:21:38.364904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:38.370103 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 490/967] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:21:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:38.371588 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b736c2d-778f-469a-8a56-fd3b119b2fdc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:38.457123 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e320429-fdaa-41f7-b236-1da02eaa6ea5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:38.479378 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:38.487758 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b736c2d-778f-469a-8a56-fd3b119b2fdc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:38.503115 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5525a-5f7d-4c3e-a16f-0db848cf07a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:38.503790 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5525a-5f7d-4c3e-a16f-0db848cf07a8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:38.504073 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5525a-5f7d-4c3e-a16f-0db848cf07a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:38.504348 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5525a-5f7d-4c3e-a16f-0db848cf07a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:38.505096 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5525a-5f7d-4c3e-a16f-0db848cf07a8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:38.505403 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5525a-5f7d-4c3e-a16f-0db848cf07a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:38.507033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5525a-5f7d-4c3e-a16f-0db848cf07a8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:38.520891 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 478/968] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Jun 29 12:21:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:38.522103 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5525a-5f7d-4c3e-a16f-0db848cf07a8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:38.533275 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3905a-7b97-42f2-a28f-3ab531e360bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:38.534787 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3905a-7b97-42f2-a28f-3ab531e360bc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:38.535661 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3905a-7b97-42f2-a28f-3ab531e360bc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:38.536128 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3905a-7b97-42f2-a28f-3ab531e360bc None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:38.537376 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3905a-7b97-42f2-a28f-3ab531e360bc None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:38.537894 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3905a-7b97-42f2-a28f-3ab531e360bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:38.541172 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3905a-7b97-42f2-a28f-3ab531e360bc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:38.568424 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:38.597756 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 491/969] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jun 29 12:21:38 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:38.638186 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5525a-5f7d-4c3e-a16f-0db848cf07a8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:38.678939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438d36d-7802-47cf-b8a5-ee1d8033a511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:38.680066 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438d36d-7802-47cf-b8a5-ee1d8033a511 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:38.680580 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438d36d-7802-47cf-b8a5-ee1d8033a511 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:38.682193 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438d36d-7802-47cf-b8a5-ee1d8033a511 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:38.694798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8438d36d-7802-47cf-b8a5-ee1d8033a511 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:38.697426 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8438d36d-7802-47cf-b8a5-ee1d8033a511 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:38.717499 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8438d36d-7802-47cf-b8a5-ee1d8033a511 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:38.717499 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:38.717499 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:38.717499 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:38.717499 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:38.717499 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:38.717499 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:38.717499 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:38.717499 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:38.717499 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:38.717499 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:38.717499 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:38.717499 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:38.717499 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:38.717499 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:38.718505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:38.718505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:38.718505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:38.718505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:38.718505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:38.718505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:38.718505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:38.718505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:38.718505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:38.718505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:38.718505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:38.718505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:38.718505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:38.718505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:38.718505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:38.718505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:38.718505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:38.719433 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:38.719433 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:38.719433 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:38.719433 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:38.721576 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 479/970] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:38.727621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3905a-7b97-42f2-a28f-3ab531e360bc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:38.841128 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3905a-7b97-42f2-a28f-3ab531e360bc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:38.879243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f926631-45d3-4601-94ce-6c851118e62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:38.879919 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f926631-45d3-4601-94ce-6c851118e62c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:38.880207 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f926631-45d3-4601-94ce-6c851118e62c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:38.880487 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f926631-45d3-4601-94ce-6c851118e62c None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:38.881227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f926631-45d3-4601-94ce-6c851118e62c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:38.881537 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f926631-45d3-4601-94ce-6c851118e62c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:38.882959 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f926631-45d3-4601-94ce-6c851118e62c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:38.895743 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 492/971] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:38.901806 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438d36d-7802-47cf-b8a5-ee1d8033a511 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:39.004527 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:39.015266 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438d36d-7802-47cf-b8a5-ee1d8033a511 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:39.049214 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a82c12-2ce6-485b-8fc6-964d6b8df0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:39.050029 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a82c12-2ce6-485b-8fc6-964d6b8df0ad None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:39.050312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a82c12-2ce6-485b-8fc6-964d6b8df0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:39.050619 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a82c12-2ce6-485b-8fc6-964d6b8df0ad None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:39.058155 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a82c12-2ce6-485b-8fc6-964d6b8df0ad None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:39.059626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a82c12-2ce6-485b-8fc6-964d6b8df0ad None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:39.075436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-16a82c12-2ce6-485b-8fc6-964d6b8df0ad None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 12:21:39.075436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:39.075436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:39.075436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:39.075436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:39.075436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:39.075436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:39.075436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:39.075436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:39.075436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:39.075436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:39.075436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:39.075436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:21:39.075436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:21:39.075436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:21:39.075436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:21:39.076418 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:39.076418 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:39.076418 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:21:39.076418 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:39.076418 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:21:39.076418 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:39.076418 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:39.076418 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:39.076418 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:39.076418 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:39.076418 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:39.076418 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:39.076418 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:39.076418 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:39.076418 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:39.076418 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:39.076418 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:39.076418 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:39.077392 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:39.077392 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:39.077392 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:21:39.077392 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:21:39.077392 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:21:39.077392 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:21:39.077392 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:21:39.077392 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:21:39.077392 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 29 12:21:39.077392 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:39.082449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 480/972] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jun 29 12:21:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:39.082602 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f926631-45d3-4601-94ce-6c851118e62c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:39.098734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:39.178990 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c981783-95aa-41bf-997a-8146d271b727 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:39.179270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c981783-95aa-41bf-997a-8146d271b727 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:39.179549 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c981783-95aa-41bf-997a-8146d271b727 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:39.180958 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 481/973] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:21:39.185826 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d723f-9037-4519-be9b-25c5e360c4c8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:39.186161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d723f-9037-4519-be9b-25c5e360c4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:39.186396 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d723f-9037-4519-be9b-25c5e360c4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:39.189714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f926631-45d3-4601-94ce-6c851118e62c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:39.203358 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-289d723f-9037-4519-be9b-25c5e360c4c8 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:39.204550 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-289d723f-9037-4519-be9b-25c5e360c4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:39.226347 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cafc910-6c87-493d-bce1-f5e5ae470c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:39.227045 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafc910-6c87-493d-bce1-f5e5ae470c00 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:39.227331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafc910-6c87-493d-bce1-f5e5ae470c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:39.227670 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cafc910-6c87-493d-bce1-f5e5ae470c00 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:39.228504 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cafc910-6c87-493d-bce1-f5e5ae470c00 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:39.228813 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cafc910-6c87-493d-bce1-f5e5ae470c00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:39.230264 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cafc910-6c87-493d-bce1-f5e5ae470c00 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:39.251663 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 493/974] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Jun 29 12:21:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:39.252674 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cafc910-6c87-493d-bce1-f5e5ae470c00 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:39.280191 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 482/975] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 12:21:39.302800 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d723f-9037-4519-be9b-25c5e360c4c8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:39.326825 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:39.372712 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cafc910-6c87-493d-bce1-f5e5ae470c00 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:39.416566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd1914-0410-4221-a736-493daa2954db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:39.416816 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:39.416962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd1914-0410-4221-a736-493daa2954db None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:39.417237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd1914-0410-4221-a736-493daa2954db None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:39.417513 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cd1914-0410-4221-a736-493daa2954db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:39.438012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d723f-9037-4519-be9b-25c5e360c4c8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:39.475477 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683d604-f4fb-4ebc-b82d-0812fa08673e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:39.477186 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683d604-f4fb-4ebc-b82d-0812fa08673e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:39.477575 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683d604-f4fb-4ebc-b82d-0812fa08673e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:39.477858 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9683d604-f4fb-4ebc-b82d-0812fa08673e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:39.478566 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9683d604-f4fb-4ebc-b82d-0812fa08673e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:39.479940 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9683d604-f4fb-4ebc-b82d-0812fa08673e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:39.493900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9683d604-f4fb-4ebc-b82d-0812fa08673e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 12:21:39.493900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:39.493900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 12:21:39.493900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 12:21:39.493900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 12:21:39.493900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 12:21:39.493900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 12:21:39.493900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:39.493900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 12:21:39.493900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:39.493900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:21:39.493900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:39.493900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:39.493900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:39.493900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:39.493900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:39.493900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:39.495479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:39.495479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:39.495479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:39.495479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:39.495479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:39.495479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:39.495479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 12:21:39.495479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 12:21:39.495479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 12:21:39.495479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 12:21:39.495479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:39.495479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:39.495479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:39.495479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:39.495479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:39.495479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:39.496408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:39.496408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:39.496408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:39.496408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:39.496408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:39.496408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:39.496408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:39.496408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:39.496408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:39.496408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:39.496408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 12:21:39.496408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:39.496408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 12:21:39.496408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:39.498681 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd1914-0410-4221-a736-493daa2954db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:39.500286 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 483/976] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jun 29 12:21:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:39.500935 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cd1914-0410-4221-a736-493daa2954db None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:39.501190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683d604-f4fb-4ebc-b82d-0812fa08673e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:39.584567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:39.620649 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9683d604-f4fb-4ebc-b82d-0812fa08673e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:39.631574 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 494/977] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:21:39 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 12:21:39.637622 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd1914-0410-4221-a736-493daa2954db None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:39.666211 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad577a3-801b-478d-b71a-80c8e6f72d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:39.669865 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad577a3-801b-478d-b71a-80c8e6f72d29 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:39.670156 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad577a3-801b-478d-b71a-80c8e6f72d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:39.670452 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad577a3-801b-478d-b71a-80c8e6f72d29 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:39.671166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad577a3-801b-478d-b71a-80c8e6f72d29 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:39.671498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad577a3-801b-478d-b71a-80c8e6f72d29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:39.672869 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad577a3-801b-478d-b71a-80c8e6f72d29 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:39.685390 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 484/978] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jun 29 12:21:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:39.691356 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad577a3-801b-478d-b71a-80c8e6f72d29 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:39.705214 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:39.759440 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cd1914-0410-4221-a736-493daa2954db None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:39.805046 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9116843-d2d3-4751-b5e2-54cbe3891e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:39.806561 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9116843-d2d3-4751-b5e2-54cbe3891e71 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:39.806845 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9116843-d2d3-4751-b5e2-54cbe3891e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:39.807127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9116843-d2d3-4751-b5e2-54cbe3891e71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:39.823210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9116843-d2d3-4751-b5e2-54cbe3891e71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:39.824677 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9116843-d2d3-4751-b5e2-54cbe3891e71 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:39.827860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad577a3-801b-478d-b71a-80c8e6f72d29 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:39.843416 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b9116843-d2d3-4751-b5e2-54cbe3891e71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:39.843416 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:39.843416 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:39.843416 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:39.843416 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:39.843416 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:39.843416 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:39.843416 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:39.843416 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:39.843416 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:39.843416 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:39.843416 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:39.843416 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:39.843416 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:39.843416 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:39.844392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:39.844392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:39.844392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:39.844392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:39.844392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:39.844392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:39.844392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:39.844392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:39.844392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:39.844392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:39.844392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:39.844392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:39.844392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:39.844392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:39.844392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:39.844392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:39.844392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:39.845324 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:39.845324 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:39.845324 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:39.845324 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:39.845324 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 495/979] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:39.850039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9116843-d2d3-4751-b5e2-54cbe3891e71 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:39.867277 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1f5b0-e03a-432f-9873-ddd698018f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:39.868726 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1f5b0-e03a-432f-9873-ddd698018f7c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:39.869011 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1f5b0-e03a-432f-9873-ddd698018f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:39.869401 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1f5b0-e03a-432f-9873-ddd698018f7c None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:39.870116 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1f5b0-e03a-432f-9873-ddd698018f7c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:39.870448 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1f5b0-e03a-432f-9873-ddd698018f7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:39.871866 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1f5b0-e03a-432f-9873-ddd698018f7c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:39.874762 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:39.878798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 485/980] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jun 29 12:21:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:39.883941 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1f5b0-e03a-432f-9873-ddd698018f7c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:39.954500 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9116843-d2d3-4751-b5e2-54cbe3891e71 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:39.960649 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:39.986249 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1f5b0-e03a-432f-9873-ddd698018f7c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:39.987963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32315ce3-0928-4d83-ba45-3a448de196df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:39.988628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32315ce3-0928-4d83-ba45-3a448de196df None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:39.988907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32315ce3-0928-4d83-ba45-3a448de196df None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:39.989183 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32315ce3-0928-4d83-ba45-3a448de196df None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:39.990022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32315ce3-0928-4d83-ba45-3a448de196df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:39.990333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32315ce3-0928-4d83-ba45-3a448de196df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:39.991802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32315ce3-0928-4d83-ba45-3a448de196df None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:40.004988 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 496/981] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:40.007175 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad636cbe-ce62-47ff-9a8a-44018351c5c9 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:40.007686 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad636cbe-ce62-47ff-9a8a-44018351c5c9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:40.008172 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad636cbe-ce62-47ff-9a8a-44018351c5c9 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:40.014346 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 497/982] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:40.014346 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad636cbe-ce62-47ff-9a8a-44018351c5c9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:40.021294 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6aea56-03bf-4050-ad8b-6aaccb8660bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:40.022220 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6aea56-03bf-4050-ad8b-6aaccb8660bc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:40.022526 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6aea56-03bf-4050-ad8b-6aaccb8660bc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:40.022807 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6aea56-03bf-4050-ad8b-6aaccb8660bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:40.023602 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6aea56-03bf-4050-ad8b-6aaccb8660bc None admin] RBAC: Adding query filter params (interface=admin, service_id=35f308f8a3b0419d94225ee1599e568b, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:40.023914 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6aea56-03bf-4050-ad8b-6aaccb8660bc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:40.025886 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6aea56-03bf-4050-ad8b-6aaccb8660bc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:40.031231 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 486/983] 10.222.0.52 () {56 vars in 1249 bytes} [Wed Jun 29 12:21:39 2022] GET /identity/v3/endpoints?service_id=35f308f8a3b0419d94225ee1599e568b&interface=admin®ion_id=RegionOne => generated 193 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:40.032459 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a3ec8-676b-45f4-a6a2-a62bee312bae None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:40.032880 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a3ec8-676b-45f4-a6a2-a62bee312bae None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:40.033007 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a3ec8-676b-45f4-a6a2-a62bee312bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:40.053898 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-192a3ec8-676b-45f4-a6a2-a62bee312bae None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:40.055105 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-192a3ec8-676b-45f4-a6a2-a62bee312bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:40.099325 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:40.118998 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad636cbe-ce62-47ff-9a8a-44018351c5c9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:40.122163 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 487/984] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:40.122994 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a3ec8-676b-45f4-a6a2-a62bee312bae None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:40.160095 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43dc06-8572-4d07-916c-ea2f53f7c597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:40.160895 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43dc06-8572-4d07-916c-ea2f53f7c597 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:40.161183 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43dc06-8572-4d07-916c-ea2f53f7c597 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:40.161465 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43dc06-8572-4d07-916c-ea2f53f7c597 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:40.168888 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f43dc06-8572-4d07-916c-ea2f53f7c597 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:40.170331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f43dc06-8572-4d07-916c-ea2f53f7c597 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:40.185198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9f43dc06-8572-4d07-916c-ea2f53f7c597 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:40.185198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:40.185198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:40.185198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:40.185198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:40.185198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:40.185198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:40.185198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:40.185198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:40.185198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:40.185198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:40.185198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:40.185198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:21:40.185198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:21:40.185198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:21:40.186145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:21:40.186145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:40.186145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:40.186145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:40.186145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:40.186145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:40.186145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:40.186145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:40.186145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:40.186145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:40.186145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:40.186145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:40.186145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:40.186145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:40.186145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:40.186145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:40.186145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:40.187449 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:21:40.187449 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:21:40.187449 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:21:40.187449 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:21:40.187449 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:21:40.187449 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:21:40.187449 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:40.187449 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:40.187449 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 498/985] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:21:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:40.187449 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43dc06-8572-4d07-916c-ea2f53f7c597 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:40.194232 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:40.227971 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a3ec8-676b-45f4-a6a2-a62bee312bae None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:40.261585 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80439982-6c9e-44d2-91af-9f4f751ec6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:40.262252 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80439982-6c9e-44d2-91af-9f4f751ec6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:40.262580 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80439982-6c9e-44d2-91af-9f4f751ec6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:40.262866 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80439982-6c9e-44d2-91af-9f4f751ec6d4 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:40.263554 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80439982-6c9e-44d2-91af-9f4f751ec6d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:40.264927 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80439982-6c9e-44d2-91af-9f4f751ec6d4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:40.270951 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 488/986] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Jun 29 12:21:40 2022] GET /identity/v3/services => generated 1497 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:21:40.271993 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80439982-6c9e-44d2-91af-9f4f751ec6d4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:40.289215 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43dc06-8572-4d07-916c-ea2f53f7c597 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:40.326563 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e10667-e494-414f-b212-3b175f42dfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:40.327244 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e10667-e494-414f-b212-3b175f42dfcc None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:40.327524 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e10667-e494-414f-b212-3b175f42dfcc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:40.327797 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e10667-e494-414f-b212-3b175f42dfcc None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:40.328540 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e10667-e494-414f-b212-3b175f42dfcc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:40.329960 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e10667-e494-414f-b212-3b175f42dfcc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:40.345824 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 499/987] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jun 29 12:21:40 2022] POST /identity/v3/services => generated 242 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:21:40.413331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80439982-6c9e-44d2-91af-9f4f751ec6d4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:40.451807 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:40.466834 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45fb056-5bdf-4a7c-be39-a9212969051f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:40.467662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45fb056-5bdf-4a7c-be39-a9212969051f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:40.467955 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45fb056-5bdf-4a7c-be39-a9212969051f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:40.468236 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45fb056-5bdf-4a7c-be39-a9212969051f None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:40.469104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45fb056-5bdf-4a7c-be39-a9212969051f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:40.469446 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45fb056-5bdf-4a7c-be39-a9212969051f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:40.470986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45fb056-5bdf-4a7c-be39-a9212969051f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:40.485368 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 489/988] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:21:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:40.492057 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e10667-e494-414f-b212-3b175f42dfcc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:40.495318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:40.601049 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e10667-e494-414f-b212-3b175f42dfcc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:40.636526 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c555fa5-2993-41d9-a178-8c78b0d787fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:40.637191 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c555fa5-2993-41d9-a178-8c78b0d787fb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:40.637482 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c555fa5-2993-41d9-a178-8c78b0d787fb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:40.637767 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c555fa5-2993-41d9-a178-8c78b0d787fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:40.638634 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c555fa5-2993-41d9-a178-8c78b0d787fb None admin] RBAC: Adding query filter params (role.id=7dedcc77e86a4fee94bf84105c0ffd71, scope.project.id=f043ade9e4f748d4adc010ffb0a47a46, user.id=8c10852a728844c6ae0a48e3772d95b7) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:40.638953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c555fa5-2993-41d9-a178-8c78b0d787fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:40.640402 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c555fa5-2993-41d9-a178-8c78b0d787fb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:40.647026 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 500/989] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:21:40 2022] GET /identity/v3/role_assignments?user.id=8c10852a728844c6ae0a48e3772d95b7&scope.project.id=f043ade9e4f748d4adc010ffb0a47a46&role.id=7dedcc77e86a4fee94bf84105c0ffd71 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:42.358896 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f39b1-e683-4f63-bc6b-f3b17d804fcb None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:42.359448 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f39b1-e683-4f63-bc6b-f3b17d804fcb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:42.359448 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f39b1-e683-4f63-bc6b-f3b17d804fcb None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:42.361079 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 490/990] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:42.366352 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39940c87-2229-43d3-9f21-dee26093f59b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:42.366767 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39940c87-2229-43d3-9f21-dee26093f59b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:42.366957 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39940c87-2229-43d3-9f21-dee26093f59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:42.385089 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-39940c87-2229-43d3-9f21-dee26093f59b None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:42.386632 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-39940c87-2229-43d3-9f21-dee26093f59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:42.432601 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959642c-2970-44e0-8128-25d8883aaba6 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:42.432894 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959642c-2970-44e0-8128-25d8883aaba6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:42.433177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8959642c-2970-44e0-8128-25d8883aaba6 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:42.434685 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 491/991] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:42.439376 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da3998-b80f-4142-8418-b583417715ed None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:42.439670 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da3998-b80f-4142-8418-b583417715ed None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:42.439952 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da3998-b80f-4142-8418-b583417715ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:42.450599 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:42.457619 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b5da3998-b80f-4142-8418-b583417715ed None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:42.458886 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b5da3998-b80f-4142-8418-b583417715ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:42.459287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 501/992] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:42 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:42.481999 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39940c87-2229-43d3-9f21-dee26093f59b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:42.530627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 492/993] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:42 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 12:21:42.552036 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da3998-b80f-4142-8418-b583417715ed None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:42.588331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39940c87-2229-43d3-9f21-dee26093f59b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:42.624390 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d395c-2c29-4698-9361-22c63200b953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:42.625121 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d395c-2c29-4698-9361-22c63200b953 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:42.625402 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d395c-2c29-4698-9361-22c63200b953 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:42.625677 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d395c-2c29-4698-9361-22c63200b953 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:42.626334 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d395c-2c29-4698-9361-22c63200b953 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:42.627730 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d395c-2c29-4698-9361-22c63200b953 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:42.640357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-2a8d395c-2c29-4698-9361-22c63200b953 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 12:21:42.640357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:42.640357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 12:21:42.640357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 12:21:42.640357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 12:21:42.640357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 12:21:42.640357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 12:21:42.640357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:42.640357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 12:21:42.640357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:42.640357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:21:42.640357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:42.640357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:42.640357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:42.640357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:42.640357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:42.640357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:42.641593 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:42.641593 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:42.641593 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:42.641593 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:42.641593 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:42.641593 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:42.641593 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 12:21:42.641593 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 12:21:42.641593 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 12:21:42.641593 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 12:21:42.641593 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:42.641593 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:42.641593 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:42.641593 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:42.641593 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:42.641593 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:42.642560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:42.642560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:42.642560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:42.642560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:42.642560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:42.642560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:42.642560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:42.642560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:42.642560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:42.642560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:42.642560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 12:21:42.642560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:42.642560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 29 12:21:42.642560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:42.642560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 502/994] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jun 29 12:21:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:42.647486 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d395c-2c29-4698-9361-22c63200b953 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:42.667727 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da3998-b80f-4142-8418-b583417715ed None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:42.687564 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:42.705598 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca07893-ad51-4cb1-99bd-7be43f577e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:42.706401 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca07893-ad51-4cb1-99bd-7be43f577e77 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:42.706734 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca07893-ad51-4cb1-99bd-7be43f577e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:42.707016 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca07893-ad51-4cb1-99bd-7be43f577e77 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:42.707675 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca07893-ad51-4cb1-99bd-7be43f577e77 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:42.709045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca07893-ad51-4cb1-99bd-7be43f577e77 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:42.715659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:42.721847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6ca07893-ad51-4cb1-99bd-7be43f577e77 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 12:21:42.721847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:42.721847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 12:21:42.721847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 12:21:42.721847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 12:21:42.721847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 12:21:42.721847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 12:21:42.721847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:42.721847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 12:21:42.721847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:42.721847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:21:42.721847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:42.721847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:42.721847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:42.721847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:42.721847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:42.721847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:42.723108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:42.723108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:42.723108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:42.723108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:42.723108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:42.723108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:42.723108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 12:21:42.723108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 12:21:42.723108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 12:21:42.723108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 12:21:42.723108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:42.723108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:42.723108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:42.723108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:42.723108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:42.723108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:42.724160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:42.724160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:42.724160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:42.724160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:42.724160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:42.724160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:42.724160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:42.724160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:42.724160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:42.724160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:42.724160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 12:21:42.724160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:42.724160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 12:21:42.724160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:42.724160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 493/995] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:21:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:42.728360 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c21bbe-6869-43e0-9891-545d647ebb5c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:42.728648 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c21bbe-6869-43e0-9891-545d647ebb5c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:42.728927 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c21bbe-6869-43e0-9891-545d647ebb5c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:42.730365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 494/996] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:42.731083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c21bbe-6869-43e0-9891-545d647ebb5c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:42.763152 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d395c-2c29-4698-9361-22c63200b953 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:42.796118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a15738-acb7-4b10-867b-d01253987079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:42.796766 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a15738-acb7-4b10-867b-d01253987079 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:42.797049 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a15738-acb7-4b10-867b-d01253987079 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:42.797329 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a15738-acb7-4b10-867b-d01253987079 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:42.798031 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a15738-acb7-4b10-867b-d01253987079 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:42.798336 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a15738-acb7-4b10-867b-d01253987079 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:42.799752 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a15738-acb7-4b10-867b-d01253987079 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:42.804443 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 503/997] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jun 29 12:21:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:42.806182 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6698ff6-cd67-4723-b3dd-69200e6102dd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:42.806498 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6698ff6-cd67-4723-b3dd-69200e6102dd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:42.806780 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6698ff6-cd67-4723-b3dd-69200e6102dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:42.822639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b6698ff6-cd67-4723-b3dd-69200e6102dd None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:42.823801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b6698ff6-cd67-4723-b3dd-69200e6102dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:42.833124 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c21bbe-6869-43e0-9891-545d647ebb5c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:42.866267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0bb3d4-9681-452e-8ca0-90c28276aaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:42.866956 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0bb3d4-9681-452e-8ca0-90c28276aaf0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:42.867238 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0bb3d4-9681-452e-8ca0-90c28276aaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:42.867513 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0bb3d4-9681-452e-8ca0-90c28276aaf0 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:42.868276 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0bb3d4-9681-452e-8ca0-90c28276aaf0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:42.868582 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0bb3d4-9681-452e-8ca0-90c28276aaf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:42.869941 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0bb3d4-9681-452e-8ca0-90c28276aaf0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:42.874732 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 495/998] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jun 29 12:21:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:42.875433 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0bb3d4-9681-452e-8ca0-90c28276aaf0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:42.889883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 504/999] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:42 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:42.890745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6698ff6-cd67-4723-b3dd-69200e6102dd None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:42.921916 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:42.948665 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:42.976563 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0bb3d4-9681-452e-8ca0-90c28276aaf0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:42.992122 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6698ff6-cd67-4723-b3dd-69200e6102dd None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:43.009235 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90161dd6-a0f6-4630-98e0-21f066e42cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:43.009899 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90161dd6-a0f6-4630-98e0-21f066e42cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:43.010183 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90161dd6-a0f6-4630-98e0-21f066e42cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:43.010481 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90161dd6-a0f6-4630-98e0-21f066e42cb5 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:43.011181 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90161dd6-a0f6-4630-98e0-21f066e42cb5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:43.011485 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90161dd6-a0f6-4630-98e0-21f066e42cb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:43.012836 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90161dd6-a0f6-4630-98e0-21f066e42cb5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:43.017541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 496/1000] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jun 29 12:21:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:43.018335 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90161dd6-a0f6-4630-98e0-21f066e42cb5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:43.032323 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27d533-39b5-497b-894a-663668f1dd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:43.032996 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27d533-39b5-497b-894a-663668f1dd59 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:43.033279 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27d533-39b5-497b-894a-663668f1dd59 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:43.033557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27d533-39b5-497b-894a-663668f1dd59 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:43.034286 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c27d533-39b5-497b-894a-663668f1dd59 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:43.034604 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c27d533-39b5-497b-894a-663668f1dd59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:43.035972 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c27d533-39b5-497b-894a-663668f1dd59 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:43.040815 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 505/1001] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jun 29 12:21:42 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:43.041560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27d533-39b5-497b-894a-663668f1dd59 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:43.120770 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90161dd6-a0f6-4630-98e0-21f066e42cb5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:43.130888 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:43.150756 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27d533-39b5-497b-894a-663668f1dd59 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:43.155982 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7663e697-39f4-4122-bd33-9fb74ba6b78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:43.156861 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663e697-39f4-4122-bd33-9fb74ba6b78e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:43.157148 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663e697-39f4-4122-bd33-9fb74ba6b78e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:43.157426 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7663e697-39f4-4122-bd33-9fb74ba6b78e None admin] PATH_INFO: `/v3/services/image` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:43.158089 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7663e697-39f4-4122-bd33-9fb74ba6b78e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:43.159489 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7663e697-39f4-4122-bd33-9fb74ba6b78e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:43.168032 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:43.171560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7663e697-39f4-4122-bd33-9fb74ba6b78e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 12:21:43.171560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:43.171560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 12:21:43.171560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 12:21:43.171560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 12:21:43.171560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 12:21:43.171560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 12:21:43.171560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:43.171560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 12:21:43.171560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:43.171560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:21:43.171560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:43.171560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:43.171560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:43.171560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:43.171560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:43.171560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:43.172848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:43.172848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:43.172848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:43.172848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:43.172848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:43.172848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:43.172848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 12:21:43.172848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 12:21:43.172848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 12:21:43.172848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 12:21:43.172848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:43.172848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:43.172848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:43.172848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:43.172848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:43.172848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:43.173691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:43.173691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:43.173691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:43.173691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:43.173691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:43.173691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:43.173691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:43.173691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:43.173691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:43.173691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:43.173691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 12:21:43.173691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:43.173691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 12:21:43.173691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:43.173691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 497/1002] 10.222.0.52 () {56 vars in 1105 bytes} [Wed Jun 29 12:21:43 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:43.174554 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7663e697-39f4-4122-bd33-9fb74ba6b78e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:43.191047 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3d820-f5c8-41d6-9a30-9dba551a619d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:43.191707 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3d820-f5c8-41d6-9a30-9dba551a619d None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:43.191995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3d820-f5c8-41d6-9a30-9dba551a619d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:43.192282 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3d820-f5c8-41d6-9a30-9dba551a619d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:43.193020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e3d820-f5c8-41d6-9a30-9dba551a619d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:43.194383 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e3d820-f5c8-41d6-9a30-9dba551a619d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:43.209846 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 506/1003] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Jun 29 12:21:43 2022] POST /identity/v3/endpoints => generated 325 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:21:43.210683 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3d820-f5c8-41d6-9a30-9dba551a619d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:43.298378 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7663e697-39f4-4122-bd33-9fb74ba6b78e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:43.336479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3d820-f5c8-41d6-9a30-9dba551a619d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:43.337451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105f916-a003-484b-8c38-e666563275d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:43.338134 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105f916-a003-484b-8c38-e666563275d3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:43.338447 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105f916-a003-484b-8c38-e666563275d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:43.338732 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9105f916-a003-484b-8c38-e666563275d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:43.339536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9105f916-a003-484b-8c38-e666563275d3 None admin] RBAC: Adding query filter params (interface=public, service_id=c159f58301af4a38bc8d8922b984b2a8, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:43.339848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9105f916-a003-484b-8c38-e666563275d3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:43.341246 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9105f916-a003-484b-8c38-e666563275d3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:43.350470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 498/1004] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Jun 29 12:21:43 2022] GET /identity/v3/endpoints?service_id=c159f58301af4a38bc8d8922b984b2a8&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:43.353616 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105f916-a003-484b-8c38-e666563275d3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:43.388887 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25dbed6-878c-4927-aea7-76f050673fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:43.389499 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25dbed6-878c-4927-aea7-76f050673fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:43.389786 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25dbed6-878c-4927-aea7-76f050673fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:43.390073 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25dbed6-878c-4927-aea7-76f050673fb0 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:43.390826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25dbed6-878c-4927-aea7-76f050673fb0 None admin] RBAC: Adding query filter params (name=image) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:43.391144 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25dbed6-878c-4927-aea7-76f050673fb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:43.392542 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25dbed6-878c-4927-aea7-76f050673fb0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:43.398455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 507/1005] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:43.404221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:43.404833 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25dbed6-878c-4927-aea7-76f050673fb0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:43.450449 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:43.476274 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9105f916-a003-484b-8c38-e666563275d3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:43.515476 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25dbed6-878c-4927-aea7-76f050673fb0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:43.516851 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa19a4-bc16-49f0-a70e-bee4e3d5218d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:43.517517 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa19a4-bc16-49f0-a70e-bee4e3d5218d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:43.517805 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa19a4-bc16-49f0-a70e-bee4e3d5218d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:43.518082 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa19a4-bc16-49f0-a70e-bee4e3d5218d None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:43.518798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa19a4-bc16-49f0-a70e-bee4e3d5218d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:43.520161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa19a4-bc16-49f0-a70e-bee4e3d5218d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:43.527283 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 499/1006] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Jun 29 12:21:43 2022] GET /identity/v3/services => generated 1727 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:21:43.557990 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984663b-76c3-4481-a2b2-ba939dbe5c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:43.558713 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984663b-76c3-4481-a2b2-ba939dbe5c68 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:43.559001 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984663b-76c3-4481-a2b2-ba939dbe5c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:43.559279 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984663b-76c3-4481-a2b2-ba939dbe5c68 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:43.560004 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984663b-76c3-4481-a2b2-ba939dbe5c68 None admin] RBAC: Adding query filter params (type=image) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:43.560315 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984663b-76c3-4481-a2b2-ba939dbe5c68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:43.561720 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984663b-76c3-4481-a2b2-ba939dbe5c68 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:43.574762 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 508/1007] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:43 2022] GET /identity/v3/services?type=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:45.253334 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda669a0-85d6-4ffd-8121-af11316fe4d4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:45.253884 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda669a0-85d6-4ffd-8121-af11316fe4d4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:45.253884 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda669a0-85d6-4ffd-8121-af11316fe4d4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:45.255419 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 500/1008] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:45.260983 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b02969-da69-4165-8cda-9fdd24bd0077 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:45.261420 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b02969-da69-4165-8cda-9fdd24bd0077 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:45.261566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b02969-da69-4165-8cda-9fdd24bd0077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:45.279852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f0b02969-da69-4165-8cda-9fdd24bd0077 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:45.281062 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f0b02969-da69-4165-8cda-9fdd24bd0077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:45.356345 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 509/1009] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:45 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:45.380080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda669a0-85d6-4ffd-8121-af11316fe4d4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:45.484343 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda669a0-85d6-4ffd-8121-af11316fe4d4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:45.485893 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:45.520191 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e991b863-34a0-44a0-8dbd-bfe57956244e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:45.520909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991b863-34a0-44a0-8dbd-bfe57956244e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:45.521194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991b863-34a0-44a0-8dbd-bfe57956244e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:45.521728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e991b863-34a0-44a0-8dbd-bfe57956244e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:45.522933 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e991b863-34a0-44a0-8dbd-bfe57956244e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:45.527939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e991b863-34a0-44a0-8dbd-bfe57956244e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:45.530178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 501/1010] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:45.535732 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b02969-da69-4165-8cda-9fdd24bd0077 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:45.568260 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:45.584863 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5ad0a-0b24-41e9-8e03-794d0979be27 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:45.585194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5ad0a-0b24-41e9-8e03-794d0979be27 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:45.585422 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5ad0a-0b24-41e9-8e03-794d0979be27 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:45.587043 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 502/1011] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:21:45.592382 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba22db9-9f9b-4460-b088-d417ec9ddb04 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:45.592681 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba22db9-9f9b-4460-b088-d417ec9ddb04 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:45.592963 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba22db9-9f9b-4460-b088-d417ec9ddb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:45.609750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-eba22db9-9f9b-4460-b088-d417ec9ddb04 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:45.610980 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-eba22db9-9f9b-4460-b088-d417ec9ddb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:45.640639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b02969-da69-4165-8cda-9fdd24bd0077 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:45.676559 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdde0ea-f9da-45d2-acf4-9f47f45a55e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:45.677216 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdde0ea-f9da-45d2-acf4-9f47f45a55e3 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:45.677498 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdde0ea-f9da-45d2-acf4-9f47f45a55e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:45.677780 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdde0ea-f9da-45d2-acf4-9f47f45a55e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:45.678528 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdde0ea-f9da-45d2-acf4-9f47f45a55e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:45.679115 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 503/1012] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:45 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 12:21:45.679967 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdde0ea-f9da-45d2-acf4-9f47f45a55e3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:45.680491 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33498e1a-8260-451e-8440-9655815fe1bb None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:45.681424 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33498e1a-8260-451e-8440-9655815fe1bb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:45.681724 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33498e1a-8260-451e-8440-9655815fe1bb None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:45.683200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 504/1013] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:45.688208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eeb9ab-39aa-4d74-aa54-096df26a2690 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:45.688503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eeb9ab-39aa-4d74-aa54-096df26a2690 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:45.688786 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eeb9ab-39aa-4d74-aa54-096df26a2690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:45.706225 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-21eeb9ab-39aa-4d74-aa54-096df26a2690 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:45.707455 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-21eeb9ab-39aa-4d74-aa54-096df26a2690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:45.717785 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 510/1014] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jun 29 12:21:45 2022] POST /identity/v3/projects => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:21:45.717785 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdde0ea-f9da-45d2-acf4-9f47f45a55e3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:45.794723 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 505/1015] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:45 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 12:21:45.809347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eeb9ab-39aa-4d74-aa54-096df26a2690 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:45.816681 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:45.836994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdde0ea-f9da-45d2-acf4-9f47f45a55e3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:45.843813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:45.880723 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04050ee4-e795-4826-b83b-a48206249451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:45.881858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04050ee4-e795-4826-b83b-a48206249451 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:45.881858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04050ee4-e795-4826-b83b-a48206249451 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:45.882065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04050ee4-e795-4826-b83b-a48206249451 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:45.882775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04050ee4-e795-4826-b83b-a48206249451 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:45.884695 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04050ee4-e795-4826-b83b-a48206249451 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:45.897355 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-04050ee4-e795-4826-b83b-a48206249451 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 12:21:45.897355 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:45.897355 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 12:21:45.897355 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 12:21:45.897355 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 12:21:45.897355 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 12:21:45.897355 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 12:21:45.897355 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:45.897355 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 12:21:45.897355 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:45.897355 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:21:45.897355 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:45.897355 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:45.897355 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:45.897355 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:45.897355 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:45.897355 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:45.898394 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:45.898394 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:45.898394 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:45.898394 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:45.898394 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:45.898394 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:45.898394 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 12:21:45.898394 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 12:21:45.898394 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 12:21:45.898394 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 12:21:45.898394 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:45.898394 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:45.898394 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:45.898394 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:45.898394 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:45.898394 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:45.900129 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:45.900129 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:45.900129 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:45.900129 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:45.900129 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:45.900129 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:45.900129 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:45.900129 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:45.900129 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:45.900129 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:45.900129 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 12:21:45.900129 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:45.900129 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 29 12:21:45.900129 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:45.905223 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 511/1016] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:21:45 2022] GET /identity/v3/services/compute => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:45.905934 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04050ee4-e795-4826-b83b-a48206249451 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:45.929137 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eeb9ab-39aa-4d74-aa54-096df26a2690 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:45.974130 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8f6fb-ef1b-4cfb-ae0b-ece04b985f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:45.974923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8f6fb-ef1b-4cfb-ae0b-ece04b985f5f None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:45.975215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8f6fb-ef1b-4cfb-ae0b-ece04b985f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:45.975764 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f8f6fb-ef1b-4cfb-ae0b-ece04b985f5f None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:45.977074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f8f6fb-ef1b-4cfb-ae0b-ece04b985f5f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:45.979334 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f8f6fb-ef1b-4cfb-ae0b-ece04b985f5f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:45.996533 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 506/1017] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jun 29 12:21:45 2022] POST /identity/v3/services => generated 242 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:21:46.030546 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04050ee4-e795-4826-b83b-a48206249451 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:46.076682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe72634-464e-4ff7-8cc8-9942604fb636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:46.077307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe72634-464e-4ff7-8cc8-9942604fb636 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:46.077596 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe72634-464e-4ff7-8cc8-9942604fb636 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:46.077904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe72634-464e-4ff7-8cc8-9942604fb636 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:46.078692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe72634-464e-4ff7-8cc8-9942604fb636 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:46.078978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe72634-464e-4ff7-8cc8-9942604fb636 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:46.080368 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe72634-464e-4ff7-8cc8-9942604fb636 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:46.091085 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 512/1018] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jun 29 12:21:45 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:46.093113 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8f6fb-ef1b-4cfb-ae0b-ece04b985f5f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:46.196522 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:46.208441 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f8f6fb-ef1b-4cfb-ae0b-ece04b985f5f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:46.245120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71031d15-e1d1-42a2-b056-056fc05a1850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:46.245794 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71031d15-e1d1-42a2-b056-056fc05a1850 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:46.246085 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71031d15-e1d1-42a2-b056-056fc05a1850 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:46.246371 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71031d15-e1d1-42a2-b056-056fc05a1850 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:46.247125 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71031d15-e1d1-42a2-b056-056fc05a1850 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:46.247440 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71031d15-e1d1-42a2-b056-056fc05a1850 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:46.248810 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71031d15-e1d1-42a2-b056-056fc05a1850 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:46.253932 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 507/1019] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jun 29 12:21:46 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:46.259936 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe72634-464e-4ff7-8cc8-9942604fb636 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:46.271760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:46.380574 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe72634-464e-4ff7-8cc8-9942604fb636 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:46.417059 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030efbb-bb60-4e89-b168-7858766e1c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:46.417726 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030efbb-bb60-4e89-b168-7858766e1c6d None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:46.418012 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030efbb-bb60-4e89-b168-7858766e1c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:46.418297 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030efbb-bb60-4e89-b168-7858766e1c6d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:46.419086 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030efbb-bb60-4e89-b168-7858766e1c6d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:46.420461 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030efbb-bb60-4e89-b168-7858766e1c6d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:46.439275 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 513/1020] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Jun 29 12:21:46 2022] POST /identity/v3/endpoints => generated 334 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:21:47.817560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468c63c-49cc-49c5-9419-e7655582750f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:47.818080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468c63c-49cc-49c5-9419-e7655582750f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:47.818158 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a468c63c-49cc-49c5-9419-e7655582750f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:47.819538 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 508/1021] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:47.824625 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b43164-3cb1-4140-82d9-2d294e6b9393 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:47.825027 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b43164-3cb1-4140-82d9-2d294e6b9393 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:47.825187 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b43164-3cb1-4140-82d9-2d294e6b9393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:47.842507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d3b43164-3cb1-4140-82d9-2d294e6b9393 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:47.843687 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d3b43164-3cb1-4140-82d9-2d294e6b9393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:47.902714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:47.919265 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 514/1022] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:47 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:47.944153 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468c63c-49cc-49c5-9419-e7655582750f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:48.045353 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a468c63c-49cc-49c5-9419-e7655582750f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:48.079671 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3361112-6233-4472-abb4-3d61061b445a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:48.080390 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3361112-6233-4472-abb4-3d61061b445a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:48.080677 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3361112-6233-4472-abb4-3d61061b445a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:48.080952 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3361112-6233-4472-abb4-3d61061b445a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:48.082124 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3361112-6233-4472-abb4-3d61061b445a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:48.083591 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3361112-6233-4472-abb4-3d61061b445a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:48.085353 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 509/1023] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:48.091505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b43164-3cb1-4140-82d9-2d294e6b9393 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:48.092405 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0afac-979f-4b1e-9518-efaef146933f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:48.092693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0afac-979f-4b1e-9518-efaef146933f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:48.092973 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d0afac-979f-4b1e-9518-efaef146933f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:48.094293 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 510/1024] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:48.099141 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60485f-069e-4fae-b0be-8bdb1b69b28d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:48.099429 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60485f-069e-4fae-b0be-8bdb1b69b28d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:48.099703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60485f-069e-4fae-b0be-8bdb1b69b28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:48.116378 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-db60485f-069e-4fae-b0be-8bdb1b69b28d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:48.117553 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-db60485f-069e-4fae-b0be-8bdb1b69b28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:48.135595 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:48.191484 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 511/1025] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:48 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 12:21:48.202041 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b43164-3cb1-4140-82d9-2d294e6b9393 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:48.215507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db60485f-069e-4fae-b0be-8bdb1b69b28d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:48.237851 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3501f8-cf06-46bd-844d-10c93c624132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:48.238464 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3501f8-cf06-46bd-844d-10c93c624132 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:48.238756 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3501f8-cf06-46bd-844d-10c93c624132 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:48.239039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3501f8-cf06-46bd-844d-10c93c624132 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:48.239765 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3501f8-cf06-46bd-844d-10c93c624132 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:48.241219 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3501f8-cf06-46bd-844d-10c93c624132 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:48.265135 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 515/1026] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 12:21:48 2022] POST /identity/v3/users => generated 285 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:21:48.335800 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db60485f-069e-4fae-b0be-8bdb1b69b28d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:48.382745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0d100-c021-40f1-9c2e-4fd94f2485f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:48.384119 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0d100-c021-40f1-9c2e-4fd94f2485f1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:48.384568 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0d100-c021-40f1-9c2e-4fd94f2485f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:48.384845 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0d100-c021-40f1-9c2e-4fd94f2485f1 None admin] PATH_INFO: `/v3/services/image` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:48.385511 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0d100-c021-40f1-9c2e-4fd94f2485f1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:48.386943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0d100-c021-40f1-9c2e-4fd94f2485f1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:48.399754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-31c0d100-c021-40f1-9c2e-4fd94f2485f1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 12:21:48.399754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:48.399754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 12:21:48.399754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 12:21:48.399754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 12:21:48.399754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 12:21:48.399754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 12:21:48.399754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:48.399754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 12:21:48.399754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:48.399754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:21:48.399754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:48.399754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:48.399754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:48.399754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:48.399754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:48.399754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:48.401033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:48.401033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:48.401033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:48.401033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:48.401033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:48.401033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:48.401033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 12:21:48.401033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 12:21:48.401033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 12:21:48.401033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 12:21:48.401033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:48.401033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:48.401033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:48.401033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:48.401033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:48.401033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:48.402102 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:48.402102 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:48.402102 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:48.402102 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:48.402102 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:48.402102 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:48.402102 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:48.402102 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:48.402102 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:48.402102 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:48.402102 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 12:21:48.402102 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:48.402102 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 12:21:48.402102 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:48.402102 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 512/1027] 10.222.0.52 () {56 vars in 1105 bytes} [Wed Jun 29 12:21:48 2022] GET /identity/v3/services/image => generated 86 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:48.407284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3501f8-cf06-46bd-844d-10c93c624132 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:48.460678 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:48.520964 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3501f8-cf06-46bd-844d-10c93c624132 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:48.554706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb463c9-f417-4fd5-9c82-9a582b0f246c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:48.556018 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb463c9-f417-4fd5-9c82-9a582b0f246c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:48.556316 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb463c9-f417-4fd5-9c82-9a582b0f246c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:48.556595 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb463c9-f417-4fd5-9c82-9a582b0f246c None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:48.557312 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb463c9-f417-4fd5-9c82-9a582b0f246c None admin] RBAC: Adding query filter params (name=image) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:48.557621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb463c9-f417-4fd5-9c82-9a582b0f246c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:48.559031 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb463c9-f417-4fd5-9c82-9a582b0f246c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:48.564023 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 516/1028] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:48.569942 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0d100-c021-40f1-9c2e-4fd94f2485f1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:48.590138 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:48.592460 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306f700-9d40-4d1d-a6a9-ca0bb88d31be None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:48.592844 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306f700-9d40-4d1d-a6a9-ca0bb88d31be None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:48.593140 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306f700-9d40-4d1d-a6a9-ca0bb88d31be None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:48.594837 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 517/1029] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:48.600391 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14bc69-1927-41a4-acfb-125ec16d1dc5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:48.600849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14bc69-1927-41a4-acfb-125ec16d1dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:48.601552 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14bc69-1927-41a4-acfb-125ec16d1dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:48.625508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-da14bc69-1927-41a4-acfb-125ec16d1dc5 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:48.627376 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-da14bc69-1927-41a4-acfb-125ec16d1dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:48.677943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0d100-c021-40f1-9c2e-4fd94f2485f1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:48.716300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8038aa0-2a2d-4c72-9e62-1399c1d8427f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:48.716969 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8038aa0-2a2d-4c72-9e62-1399c1d8427f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:48.717257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8038aa0-2a2d-4c72-9e62-1399c1d8427f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:48.717536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8038aa0-2a2d-4c72-9e62-1399c1d8427f None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:48.718457 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8038aa0-2a2d-4c72-9e62-1399c1d8427f None admin] RBAC: Adding query filter params (type=image) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:48.718591 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 518/1030] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:48 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 12:21:48.718692 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8038aa0-2a2d-4c72-9e62-1399c1d8427f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:48.720170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8038aa0-2a2d-4c72-9e62-1399c1d8427f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:48.731730 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 513/1031] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:48 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:48.733010 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14bc69-1927-41a4-acfb-125ec16d1dc5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:48.744919 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8038aa0-2a2d-4c72-9e62-1399c1d8427f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:48.775510 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:48.834587 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:48.838606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14bc69-1927-41a4-acfb-125ec16d1dc5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:48.859145 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8038aa0-2a2d-4c72-9e62-1399c1d8427f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:48.873984 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5f7b8-0bf1-45d0-aa5b-f798ba540e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:48.874676 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5f7b8-0bf1-45d0-aa5b-f798ba540e4a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:48.874970 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5f7b8-0bf1-45d0-aa5b-f798ba540e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:48.875254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5f7b8-0bf1-45d0-aa5b-f798ba540e4a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:48.876051 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5f7b8-0bf1-45d0-aa5b-f798ba540e4a None admin] RBAC: Adding query filter params (interface=public, service_id=4d55ddbeae1e4450b05ff600c6c30686, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:48.876368 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5f7b8-0bf1-45d0-aa5b-f798ba540e4a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:48.877744 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5f7b8-0bf1-45d0-aa5b-f798ba540e4a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:48.883209 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 519/1032] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Jun 29 12:21:48 2022] GET /identity/v3/endpoints?service_id=4d55ddbeae1e4450b05ff600c6c30686&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:48.889089 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5f7b8-0bf1-45d0-aa5b-f798ba540e4a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:48.901025 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9842c-11e6-43a7-a112-2211ee951145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:48.902147 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9842c-11e6-43a7-a112-2211ee951145 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:48.902667 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9842c-11e6-43a7-a112-2211ee951145 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:48.902877 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e9842c-11e6-43a7-a112-2211ee951145 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:48.913348 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e9842c-11e6-43a7-a112-2211ee951145 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:21:48.921966 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-97e9842c-11e6-43a7-a112-2211ee951145 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 12:21:48.921966 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:48.921966 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:48.921966 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:48.921966 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:48.921966 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:48.921966 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:48.921966 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:48.921966 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:48.921966 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:48.921966 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:48.921966 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:48.921966 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:21:48.921966 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:21:48.921966 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:21:48.921966 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:21:48.923198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:21:48.923198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:21:48.923198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:21:48.923198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:21:48.923198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:48.923198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:48.923198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:48.923198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:48.923198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:48.923198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:48.923198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:48.923198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:48.923198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:48.923198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:48.923198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:48.923198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:48.927444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:48.927444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:48.927444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:48.927444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:48.927444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:21:48.927444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:21:48.927444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:21:48.927444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:21:48.927444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:21:48.927444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:21:48.927444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 12:21:48.927444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:48.927444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 514/1033] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Jun 29 12:21:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:48.931293 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9842c-11e6-43a7-a112-2211ee951145 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:49.002932 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5f7b8-0bf1-45d0-aa5b-f798ba540e4a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:49.005823 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:49.044362 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab98000-8050-4463-9482-fd677dd3fb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:49.045033 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab98000-8050-4463-9482-fd677dd3fb57 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:49.045323 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab98000-8050-4463-9482-fd677dd3fb57 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:49.045603 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab98000-8050-4463-9482-fd677dd3fb57 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:49.046009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e9842c-11e6-43a7-a112-2211ee951145 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:49.047144 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab98000-8050-4463-9482-fd677dd3fb57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:49.048540 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab98000-8050-4463-9482-fd677dd3fb57 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:49.055330 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 520/1034] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Jun 29 12:21:48 2022] GET /identity/v3/services => generated 1957 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:21:49.092569 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534f3fc-1faa-4273-bb24-f08cd3e2405a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:49.093262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534f3fc-1faa-4273-bb24-f08cd3e2405a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:49.093549 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534f3fc-1faa-4273-bb24-f08cd3e2405a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:49.093830 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534f3fc-1faa-4273-bb24-f08cd3e2405a None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:49.094661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5534f3fc-1faa-4273-bb24-f08cd3e2405a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:49.094975 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5534f3fc-1faa-4273-bb24-f08cd3e2405a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:49.096380 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5534f3fc-1faa-4273-bb24-f08cd3e2405a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:49.111169 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 515/1035] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Jun 29 12:21:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:49.112523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab98000-8050-4463-9482-fd677dd3fb57 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:49.223993 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab98000-8050-4463-9482-fd677dd3fb57 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:49.260333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcfe97-6abc-4ef8-b1b9-d6cafc42976e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:49.261497 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcfe97-6abc-4ef8-b1b9-d6cafc42976e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:49.261989 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcfe97-6abc-4ef8-b1b9-d6cafc42976e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:49.262559 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dcfe97-6abc-4ef8-b1b9-d6cafc42976e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:49.273291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dcfe97-6abc-4ef8-b1b9-d6cafc42976e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:49.275604 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dcfe97-6abc-4ef8-b1b9-d6cafc42976e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:49.293417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-84dcfe97-6abc-4ef8-b1b9-d6cafc42976e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:49.293417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:49.293417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:49.293417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:49.293417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:49.293417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:49.293417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:49.293417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:49.293417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:49.293417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:49.293417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:49.293417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:49.293417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:49.293417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:49.293417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:49.294342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:49.294342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:49.294342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:49.294342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:49.294342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:49.294342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:49.294342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:49.294342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:49.294342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:49.294342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:49.294342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:49.294342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:49.294342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:49.294342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:49.294342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:49.294342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:49.294342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:49.295438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:49.295438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:49.295438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:49.295438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:49.295866 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 521/1036] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:49.301565 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534f3fc-1faa-4273-bb24-f08cd3e2405a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:49.304599 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:49.430140 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534f3fc-1faa-4273-bb24-f08cd3e2405a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:49.464342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98157c84-79d9-4f8d-9f4a-ed8ee960d5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:49.465012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98157c84-79d9-4f8d-9f4a-ed8ee960d5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:49.465297 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98157c84-79d9-4f8d-9f4a-ed8ee960d5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:49.465573 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98157c84-79d9-4f8d-9f4a-ed8ee960d5b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:49.466297 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98157c84-79d9-4f8d-9f4a-ed8ee960d5b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:49.466643 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98157c84-79d9-4f8d-9f4a-ed8ee960d5b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:49.468031 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98157c84-79d9-4f8d-9f4a-ed8ee960d5b5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:49.486178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 516/1037] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:49.487529 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcfe97-6abc-4ef8-b1b9-d6cafc42976e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:49.510978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:49.593277 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dcfe97-6abc-4ef8-b1b9-d6cafc42976e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:49.627148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864dba38-7541-4121-a7e5-ecae02540ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:49.627859 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864dba38-7541-4121-a7e5-ecae02540ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:49.628150 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864dba38-7541-4121-a7e5-ecae02540ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:49.628432 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864dba38-7541-4121-a7e5-ecae02540ee9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:49.636184 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864dba38-7541-4121-a7e5-ecae02540ee9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:49.637630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864dba38-7541-4121-a7e5-ecae02540ee9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:49.653134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-864dba38-7541-4121-a7e5-ecae02540ee9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 12:21:49.653134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:49.653134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:49.653134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:49.653134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:49.653134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:49.653134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:49.653134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:49.653134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:49.653134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:49.653134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:49.653134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:49.653134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:21:49.653134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:21:49.653134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:21:49.653134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:21:49.654104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:49.654104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:49.654104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:21:49.654104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:49.654104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:21:49.654104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:49.654104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:49.654104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:49.654104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:49.654104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:49.654104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:49.654104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:49.654104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:49.654104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:49.654104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:49.654104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:49.654104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:49.654104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:49.655107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:49.655107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:49.655107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:21:49.655107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:21:49.655107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:21:49.655107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:21:49.655107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:21:49.655107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:21:49.655107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 12:21:49.655107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:49.658834 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 522/1038] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Jun 29 12:21:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:49.660061 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98157c84-79d9-4f8d-9f4a-ed8ee960d5b5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:49.725804 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:49.767153 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98157c84-79d9-4f8d-9f4a-ed8ee960d5b5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:49.801885 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5eeb9-2d66-491d-a998-fef88980728e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:49.802599 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5eeb9-2d66-491d-a998-fef88980728e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:49.802887 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5eeb9-2d66-491d-a998-fef88980728e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:49.803166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5eeb9-2d66-491d-a998-fef88980728e None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:49.804002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5eeb9-2d66-491d-a998-fef88980728e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:49.804315 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5eeb9-2d66-491d-a998-fef88980728e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:49.805787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5eeb9-2d66-491d-a998-fef88980728e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:49.827822 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 517/1039] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Jun 29 12:21:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:49.829291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864dba38-7541-4121-a7e5-ecae02540ee9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:49.911910 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:49.932965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864dba38-7541-4121-a7e5-ecae02540ee9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:49.966371 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50e56f-831a-420d-8beb-22d274698970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:49.967044 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50e56f-831a-420d-8beb-22d274698970 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:49.967325 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50e56f-831a-420d-8beb-22d274698970 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:49.967599 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50e56f-831a-420d-8beb-22d274698970 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:50.033703 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50e56f-831a-420d-8beb-22d274698970 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:50.035177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50e56f-831a-420d-8beb-22d274698970 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:50.112385 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:50.134239 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 523/1040] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:21:49 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 12:21:50.139525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5eeb9-2d66-491d-a998-fef88980728e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:50.241279 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5eeb9-2d66-491d-a998-fef88980728e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:50.262063 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:50.280648 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ea8978-8219-4ba6-9705-c55624f5b801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:50.281544 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea8978-8219-4ba6-9705-c55624f5b801 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:50.281894 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea8978-8219-4ba6-9705-c55624f5b801 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:50.282222 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ea8978-8219-4ba6-9705-c55624f5b801 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:50.293499 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ea8978-8219-4ba6-9705-c55624f5b801 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:50.295042 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ea8978-8219-4ba6-9705-c55624f5b801 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:50.312884 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f0ea8978-8219-4ba6-9705-c55624f5b801 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:50.312884 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:50.312884 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:50.312884 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:50.312884 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:50.312884 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:50.312884 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:50.312884 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:50.312884 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:50.312884 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:50.312884 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:50.312884 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:50.312884 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:50.312884 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:50.312884 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:50.314021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:50.314021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:50.314021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:50.314021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:50.314021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:50.314021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:50.314021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:50.314021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:50.314021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:50.314021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:50.314021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:50.314021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:50.314021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:50.314021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:50.314021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:50.314021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:50.314021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:50.315351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:50.315351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:50.315351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:50.315351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:50.316340 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 518/1041] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:50.321167 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50e56f-831a-420d-8beb-22d274698970 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:50.429770 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50e56f-831a-420d-8beb-22d274698970 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:50.431172 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948703d-1f98-4b62-be60-3837a6ee3adf None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:50.431468 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948703d-1f98-4b62-be60-3837a6ee3adf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:50.431757 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948703d-1f98-4b62-be60-3837a6ee3adf None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:50.433402 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 519/1042] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:50.438395 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad761b1-1d25-4778-9d89-a8d64ca635eb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:50.438729 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad761b1-1d25-4778-9d89-a8d64ca635eb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:50.439018 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad761b1-1d25-4778-9d89-a8d64ca635eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:50.455839 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fad761b1-1d25-4778-9d89-a8d64ca635eb None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:50.457030 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fad761b1-1d25-4778-9d89-a8d64ca635eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:50.470307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd99607-a6a7-495f-b43e-7c33e7e1e58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:50.471108 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd99607-a6a7-495f-b43e-7c33e7e1e58b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:50.471394 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd99607-a6a7-495f-b43e-7c33e7e1e58b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:50.472031 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd99607-a6a7-495f-b43e-7c33e7e1e58b None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:50.472500 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd99607-a6a7-495f-b43e-7c33e7e1e58b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:50.472817 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd99607-a6a7-495f-b43e-7c33e7e1e58b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:50.474314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd99607-a6a7-495f-b43e-7c33e7e1e58b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:50.487501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 524/1043] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:50.493864 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd99607-a6a7-495f-b43e-7c33e7e1e58b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:50.529611 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 520/1044] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:50 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 12:21:50.536475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:50.553288 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad761b1-1d25-4778-9d89-a8d64ca635eb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:50.597253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd99607-a6a7-495f-b43e-7c33e7e1e58b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:50.635411 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65304c68-7c7d-496e-91ba-935b4ed6f197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:50.636210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65304c68-7c7d-496e-91ba-935b4ed6f197 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:50.636500 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65304c68-7c7d-496e-91ba-935b4ed6f197 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:50.636783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65304c68-7c7d-496e-91ba-935b4ed6f197 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:50.644857 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65304c68-7c7d-496e-91ba-935b4ed6f197 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:50.646310 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65304c68-7c7d-496e-91ba-935b4ed6f197 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:50.661160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-65304c68-7c7d-496e-91ba-935b4ed6f197 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:50.661160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:50.661160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:50.661160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:50.661160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:50.661160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:50.661160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:50.661160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:50.661160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:50.661160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:50.661160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:50.661160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:50.661160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:21:50.661160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:21:50.661160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:21:50.662077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:21:50.662077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:50.662077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:50.662077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:50.662077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:50.662077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:50.662077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:50.662077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:50.662077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:50.662077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:50.662077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:50.662077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:50.662077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:50.662077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:50.662077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:50.662077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:50.662077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:50.663001 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:21:50.663001 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:21:50.663001 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:21:50.663001 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:21:50.663001 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:21:50.663001 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:21:50.663001 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:50.663001 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:50.663001 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 525/1045] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:21:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:50.666726 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad761b1-1d25-4778-9d89-a8d64ca635eb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:50.668491 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:50.669731 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65304c68-7c7d-496e-91ba-935b4ed6f197 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:50.705234 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0edc2-3492-4051-9692-1627f3edd426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:50.706290 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0edc2-3492-4051-9692-1627f3edd426 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:50.706610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0edc2-3492-4051-9692-1627f3edd426 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:50.706894 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0edc2-3492-4051-9692-1627f3edd426 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:50.718068 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0edc2-3492-4051-9692-1627f3edd426 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:21:50.725703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7af0edc2-3492-4051-9692-1627f3edd426 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:21:50.725703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:50.725703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:50.725703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:50.725703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:50.725703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:50.725703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:50.725703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:50.725703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:50.725703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:50.725703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:50.725703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:50.725703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:21:50.725703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:21:50.725703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:21:50.725703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:21:50.726678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:21:50.726678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:21:50.726678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:21:50.726678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:21:50.726678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:50.726678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:50.726678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:50.726678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:50.726678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:50.726678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:50.726678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:50.726678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:50.726678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:50.726678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:50.726678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:50.726678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:50.727609 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:50.727609 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:50.727609 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:50.727609 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:50.727609 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:21:50.727609 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:21:50.727609 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:21:50.727609 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:21:50.727609 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:21:50.727609 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:21:50.727609 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:21:50.727609 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:50.731974 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 521/1046] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:21:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:50.732852 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0edc2-3492-4051-9692-1627f3edd426 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:50.777911 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:50.781650 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65304c68-7c7d-496e-91ba-935b4ed6f197 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:50.825506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4259de3-35cb-4c93-a76d-3d30f5fb20da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:50.826173 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4259de3-35cb-4c93-a76d-3d30f5fb20da None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:50.827000 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4259de3-35cb-4c93-a76d-3d30f5fb20da None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:50.827296 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4259de3-35cb-4c93-a76d-3d30f5fb20da None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:50.828375 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4259de3-35cb-4c93-a76d-3d30f5fb20da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:50.828695 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4259de3-35cb-4c93-a76d-3d30f5fb20da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:50.831039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4259de3-35cb-4c93-a76d-3d30f5fb20da None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:50.839079 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0edc2-3492-4051-9692-1627f3edd426 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:50.851495 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 526/1047] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:21:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:50.853253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4259de3-35cb-4c93-a76d-3d30f5fb20da None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:50.874721 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a9c108-20c2-49ab-acb8-ecd09cdc81fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:50.875295 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9c108-20c2-49ab-acb8-ecd09cdc81fa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:50.875579 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9c108-20c2-49ab-acb8-ecd09cdc81fa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:50.875854 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9c108-20c2-49ab-acb8-ecd09cdc81fa None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:50.876590 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a9c108-20c2-49ab-acb8-ecd09cdc81fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:50.876895 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a9c108-20c2-49ab-acb8-ecd09cdc81fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:50.878247 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a9c108-20c2-49ab-acb8-ecd09cdc81fa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:50.887870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 522/1048] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:21:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:50.893035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a9c108-20c2-49ab-acb8-ecd09cdc81fa None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:50.921066 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:50.968694 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4259de3-35cb-4c93-a76d-3d30f5fb20da None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:51.000315 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a9c108-20c2-49ab-acb8-ecd09cdc81fa None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:51.004719 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb0d3e-db35-47cf-b595-7f9d79ebab2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:51.005570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb0d3e-db35-47cf-b595-7f9d79ebab2d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:51.006025 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb0d3e-db35-47cf-b595-7f9d79ebab2d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:51.006479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb0d3e-db35-47cf-b595-7f9d79ebab2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:51.007570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb0d3e-db35-47cf-b595-7f9d79ebab2d None admin] RBAC: Adding query filter params (role.id=7dedcc77e86a4fee94bf84105c0ffd71, scope.project.id=f043ade9e4f748d4adc010ffb0a47a46, user.id=3ac7391a3c41499d94e5f97478797313) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:51.008031 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb0d3e-db35-47cf-b595-7f9d79ebab2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:51.010270 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb0d3e-db35-47cf-b595-7f9d79ebab2d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:51.022821 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 527/1049] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:21:50 2022] GET /identity/v3/role_assignments?user.id=3ac7391a3c41499d94e5f97478797313&scope.project.id=f043ade9e4f748d4adc010ffb0a47a46&role.id=7dedcc77e86a4fee94bf84105c0ffd71 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:51.040094 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc07b6-284f-4f3d-bb9b-e14f3dfd9235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:51.041055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc07b6-284f-4f3d-bb9b-e14f3dfd9235 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:51.041346 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc07b6-284f-4f3d-bb9b-e14f3dfd9235 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:51.041631 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc07b6-284f-4f3d-bb9b-e14f3dfd9235 None admin] PATH_INFO: `/v3/users/ad699d057c6c43a4a7267007208ce0a4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:51.058137 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc07b6-284f-4f3d-bb9b-e14f3dfd9235 None admin] RBAC: Authorizing `identity:get_user(user_id=ad699d057c6c43a4a7267007208ce0a4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:51.059669 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc07b6-284f-4f3d-bb9b-e14f3dfd9235 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:51.061428 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 523/1050] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:21:50 2022] GET /identity/v3/users/ad699d057c6c43a4a7267007208ce0a4 => generated 285 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:51.067361 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb0d3e-db35-47cf-b595-7f9d79ebab2d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:51.097020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:51.154357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177d86a-d759-4331-8604-e2f7a623e974 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:51.154914 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177d86a-d759-4331-8604-e2f7a623e974 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:51.155129 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8177d86a-d759-4331-8604-e2f7a623e974 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:51.156683 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 524/1051] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:21:51.162035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8eb7b-2339-4fcb-9177-e496f017920f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:51.162334 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8eb7b-2339-4fcb-9177-e496f017920f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:51.162660 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c8eb7b-2339-4fcb-9177-e496f017920f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:51.180821 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb0d3e-db35-47cf-b595-7f9d79ebab2d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:51.183334 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-08c8eb7b-2339-4fcb-9177-e496f017920f None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:51.184529 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-08c8eb7b-2339-4fcb-9177-e496f017920f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:51.218639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bbd8bc-afa1-421f-9ca0-99753e412af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:51.219291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbd8bc-afa1-421f-9ca0-99753e412af0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:51.219575 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbd8bc-afa1-421f-9ca0-99753e412af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:51.219851 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bbd8bc-afa1-421f-9ca0-99753e412af0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:51.259326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 525/1052] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:51 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 12:21:51.280032 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c8eb7b-2339-4fcb-9177-e496f017920f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:51.290546 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bbd8bc-afa1-421f-9ca0-99753e412af0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:51.292282 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bbd8bc-afa1-421f-9ca0-99753e412af0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:51.294281 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:51.317032 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:51.395927 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c8eb7b-2339-4fcb-9177-e496f017920f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:51.407650 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 528/1053] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:21:51 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 12:21:51.412759 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bbd8bc-afa1-421f-9ca0-99753e412af0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:51.431918 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8271742-6624-498d-97e3-7a8c5a1bf45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:51.433035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8271742-6624-498d-97e3-7a8c5a1bf45f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:51.433567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8271742-6624-498d-97e3-7a8c5a1bf45f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:51.434037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8271742-6624-498d-97e3-7a8c5a1bf45f None admin] PATH_INFO: `/v3/services/image` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:51.434930 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8271742-6624-498d-97e3-7a8c5a1bf45f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:51.436486 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8271742-6624-498d-97e3-7a8c5a1bf45f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:51.449081 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f8271742-6624-498d-97e3-7a8c5a1bf45f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 12:21:51.449081 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:51.449081 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 12:21:51.449081 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 12:21:51.449081 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 12:21:51.449081 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 12:21:51.449081 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 12:21:51.449081 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:51.449081 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 12:21:51.449081 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:51.449081 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:21:51.449081 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:51.449081 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:51.449081 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:51.449081 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:51.449081 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:51.449081 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:51.450152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:51.450152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:51.450152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:51.450152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:51.450152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:51.450152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:51.450152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 12:21:51.450152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 12:21:51.450152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 12:21:51.450152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 12:21:51.450152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:51.450152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:51.450152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:51.450152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:51.450152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:51.450152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:51.451208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:51.451208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:51.451208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:51.451208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:51.451208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:51.451208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:51.451208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:51.451208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:51.451208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:51.451208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:51.451208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 12:21:51.451208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:21:51.451208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 29 12:21:51.451208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:51.454273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 526/1054] 10.222.0.52 () {56 vars in 1105 bytes} [Wed Jun 29 12:21:51 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:51.459797 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8271742-6624-498d-97e3-7a8c5a1bf45f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:51.519119 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bbd8bc-afa1-421f-9ca0-99753e412af0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:51.543474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:51.547932 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:51.558535 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a46d0-5d74-4139-9ab8-79405bd2a167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:51.559389 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a46d0-5d74-4139-9ab8-79405bd2a167 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:51.559669 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a46d0-5d74-4139-9ab8-79405bd2a167 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:51.559944 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a46d0-5d74-4139-9ab8-79405bd2a167 None admin] PATH_INFO: `/v3/projects/9f8f17b758264cf38b4c2b5fe48194da` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:51.561466 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a46d0-5d74-4139-9ab8-79405bd2a167 None admin] RBAC: Authorizing `identity:get_project(project_id=9f8f17b758264cf38b4c2b5fe48194da)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:51.562933 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a46d0-5d74-4139-9ab8-79405bd2a167 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:51.564817 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 529/1055] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:21:51 2022] GET /identity/v3/projects/9f8f17b758264cf38b4c2b5fe48194da => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:51.570640 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a46d0-5d74-4139-9ab8-79405bd2a167 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:51.575393 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8271742-6624-498d-97e3-7a8c5a1bf45f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:51.611620 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36890d66-5334-4f4f-932d-65fae4fd6f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:51.612288 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36890d66-5334-4f4f-932d-65fae4fd6f2a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:51.612572 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36890d66-5334-4f4f-932d-65fae4fd6f2a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:51.612845 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36890d66-5334-4f4f-932d-65fae4fd6f2a None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:51.613547 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36890d66-5334-4f4f-932d-65fae4fd6f2a None admin] RBAC: Adding query filter params (name=image) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:51.613852 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36890d66-5334-4f4f-932d-65fae4fd6f2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:51.615653 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36890d66-5334-4f4f-932d-65fae4fd6f2a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:51.622105 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 527/1056] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:51.627619 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36890d66-5334-4f4f-932d-65fae4fd6f2a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:51.679117 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a46d0-5d74-4139-9ab8-79405bd2a167 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:51.715482 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f20cc8-9916-49ea-bf4e-e51e47723dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:51.716141 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f20cc8-9916-49ea-bf4e-e51e47723dbe None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:51.716428 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f20cc8-9916-49ea-bf4e-e51e47723dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:51.716713 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f20cc8-9916-49ea-bf4e-e51e47723dbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:51.717652 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f20cc8-9916-49ea-bf4e-e51e47723dbe None admin] RBAC: Adding query filter params (role.id=3a7f4bf556344407a6d9540c89d13641, scope.project.id=9f8f17b758264cf38b4c2b5fe48194da, user.id=ad699d057c6c43a4a7267007208ce0a4) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:51.717970 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f20cc8-9916-49ea-bf4e-e51e47723dbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:51.719569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f20cc8-9916-49ea-bf4e-e51e47723dbe None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:51.734455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 530/1057] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:21:51 2022] GET /identity/v3/role_assignments?user.id=ad699d057c6c43a4a7267007208ce0a4&scope.project.id=9f8f17b758264cf38b4c2b5fe48194da&role.id=3a7f4bf556344407a6d9540c89d13641 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:51.734671 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36890d66-5334-4f4f-932d-65fae4fd6f2a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:51.772410 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895cd4f1-e733-486e-a6c0-e15b06844d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:51.773097 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cd4f1-e733-486e-a6c0-e15b06844d88 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:51.773369 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cd4f1-e733-486e-a6c0-e15b06844d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:51.773655 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895cd4f1-e733-486e-a6c0-e15b06844d88 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:51.774402 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895cd4f1-e733-486e-a6c0-e15b06844d88 None admin] RBAC: Adding query filter params (type=image) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:51.774745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895cd4f1-e733-486e-a6c0-e15b06844d88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:51.776393 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895cd4f1-e733-486e-a6c0-e15b06844d88 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:51.786464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 528/1058] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:51 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:51.788579 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f20cc8-9916-49ea-bf4e-e51e47723dbe None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:51.853566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:51.900337 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f20cc8-9916-49ea-bf4e-e51e47723dbe None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:51.932737 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fa89d-fc48-4efa-9ab5-0717f125cdcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:51.933390 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fa89d-fc48-4efa-9ab5-0717f125cdcc None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:51.933670 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fa89d-fc48-4efa-9ab5-0717f125cdcc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:51.933944 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fa89d-fc48-4efa-9ab5-0717f125cdcc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:51.934701 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35fa89d-fc48-4efa-9ab5-0717f125cdcc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:51.936070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35fa89d-fc48-4efa-9ab5-0717f125cdcc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:51.956265 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 531/1059] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Jun 29 12:21:51 2022] POST /identity/v3/endpoints => generated 327 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:21:53.046923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd3813-8d50-472d-a7f8-912fc29060f3 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:53.047440 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd3813-8d50-472d-a7f8-912fc29060f3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:53.047546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd3813-8d50-472d-a7f8-912fc29060f3 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:53.048969 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 529/1060] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:53.054014 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970d241-1452-4073-b95d-0fc0ef217d4f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:53.054390 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970d241-1452-4073-b95d-0fc0ef217d4f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:53.054611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970d241-1452-4073-b95d-0fc0ef217d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:53.072104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8970d241-1452-4073-b95d-0fc0ef217d4f None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:53.073332 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8970d241-1452-4073-b95d-0fc0ef217d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:53.148571 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 532/1061] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:53.171387 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd3813-8d50-472d-a7f8-912fc29060f3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:53.176259 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:53.277282 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd3813-8d50-472d-a7f8-912fc29060f3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:53.310066 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37968d2-1020-4ce0-8613-8cc50722467b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:53.310839 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37968d2-1020-4ce0-8613-8cc50722467b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:53.311128 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37968d2-1020-4ce0-8613-8cc50722467b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:53.311404 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37968d2-1020-4ce0-8613-8cc50722467b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:53.322580 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37968d2-1020-4ce0-8613-8cc50722467b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:21:53.330717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a37968d2-1020-4ce0-8613-8cc50722467b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 12:21:53.330717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:53.330717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:53.330717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:53.330717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:53.330717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:53.330717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:53.330717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:53.330717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:53.330717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:53.330717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:53.330717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:53.330717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:21:53.330717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:21:53.330717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:21:53.330717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:21:53.331644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:21:53.331644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:21:53.331644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:21:53.331644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:21:53.331644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:53.331644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:53.331644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:53.331644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:53.331644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:53.331644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:53.331644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:53.331644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:53.331644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:53.331644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:53.331644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:53.331644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:53.332492 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:53.332492 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:53.332492 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:53.332492 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:53.332492 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:21:53.332492 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:21:53.332492 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:21:53.332492 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:21:53.332492 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:21:53.332492 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:21:53.332492 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 12:21:53.332492 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:53.334744 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 530/1062] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Jun 29 12:21:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:53.338734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8970d241-1452-4073-b95d-0fc0ef217d4f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:53.388519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:53.449777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8970d241-1452-4073-b95d-0fc0ef217d4f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:53.484137 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa93a5c-c17a-4221-ab7b-7340a1d6441f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:53.484804 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa93a5c-c17a-4221-ab7b-7340a1d6441f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:53.485087 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa93a5c-c17a-4221-ab7b-7340a1d6441f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:53.485361 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa93a5c-c17a-4221-ab7b-7340a1d6441f None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:53.486100 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa93a5c-c17a-4221-ab7b-7340a1d6441f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:53.486404 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa93a5c-c17a-4221-ab7b-7340a1d6441f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:53.487788 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa93a5c-c17a-4221-ab7b-7340a1d6441f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:53.500429 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 533/1063] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Jun 29 12:21:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:53.502080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37968d2-1020-4ce0-8613-8cc50722467b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:53.566661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:53.605439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37968d2-1020-4ce0-8613-8cc50722467b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:53.640307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bdfd36-bbb6-4860-8bde-74f0fa564e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:53.641106 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdfd36-bbb6-4860-8bde-74f0fa564e41 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:53.641393 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdfd36-bbb6-4860-8bde-74f0fa564e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:53.641668 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdfd36-bbb6-4860-8bde-74f0fa564e41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:53.652331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bdfd36-bbb6-4860-8bde-74f0fa564e41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:53.653772 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bdfd36-bbb6-4860-8bde-74f0fa564e41 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:53.671291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d4bdfd36-bbb6-4860-8bde-74f0fa564e41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:53.671291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:53.671291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:53.671291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:53.671291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:53.671291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:53.671291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:53.671291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:53.671291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:53.671291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:53.671291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:53.671291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:53.671291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:53.671291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:53.671291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:53.672307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:53.672307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:53.672307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:53.672307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:53.672307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:53.672307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:53.672307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:53.672307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:53.672307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:53.672307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:53.672307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:53.672307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:53.672307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:53.672307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:53.672307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:53.672307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:53.672307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:53.673203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:53.673203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:53.673203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:53.673203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:53.677489 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 531/1064] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:53.678949 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa93a5c-c17a-4221-ab7b-7340a1d6441f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:53.777032 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9de3d-00fe-41d4-a7b2-e538bc903127 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:53.777362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9de3d-00fe-41d4-a7b2-e538bc903127 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:53.777604 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa9de3d-00fe-41d4-a7b2-e538bc903127 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:53.779025 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 532/1065] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:21:53.783474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:53.784510 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae159864-9b6d-419d-93bf-1b62058676c0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:53.784803 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae159864-9b6d-419d-93bf-1b62058676c0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:53.785098 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae159864-9b6d-419d-93bf-1b62058676c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:53.787008 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa93a5c-c17a-4221-ab7b-7340a1d6441f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:53.802008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ae159864-9b6d-419d-93bf-1b62058676c0 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:53.803254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ae159864-9b6d-419d-93bf-1b62058676c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:53.824953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1fad02-c837-4cf5-87b2-c0c5ed3f6275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:53.825548 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fad02-c837-4cf5-87b2-c0c5ed3f6275 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:53.825833 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fad02-c837-4cf5-87b2-c0c5ed3f6275 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:53.826110 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fad02-c837-4cf5-87b2-c0c5ed3f6275 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:53.826860 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1fad02-c837-4cf5-87b2-c0c5ed3f6275 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:53.827170 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1fad02-c837-4cf5-87b2-c0c5ed3f6275 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:53.828543 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1fad02-c837-4cf5-87b2-c0c5ed3f6275 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:53.840694 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 534/1066] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:53.846549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1fad02-c837-4cf5-87b2-c0c5ed3f6275 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:53.881715 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 533/1067] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 12:21:53.904893 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae159864-9b6d-419d-93bf-1b62058676c0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:53.952405 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1fad02-c837-4cf5-87b2-c0c5ed3f6275 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:53.954350 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:53.988436 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be7ed90-dc98-44a2-8347-5e1bdf0b3279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:53.989233 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7ed90-dc98-44a2-8347-5e1bdf0b3279 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:53.989523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7ed90-dc98-44a2-8347-5e1bdf0b3279 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:53.990073 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7ed90-dc98-44a2-8347-5e1bdf0b3279 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:53.997725 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be7ed90-dc98-44a2-8347-5e1bdf0b3279 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:53.999203 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be7ed90-dc98-44a2-8347-5e1bdf0b3279 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:54.013817 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-3be7ed90-dc98-44a2-8347-5e1bdf0b3279 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 12:21:54.013817 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:54.013817 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:54.013817 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:54.013817 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:54.013817 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:54.013817 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:54.013817 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:54.013817 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:54.013817 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:54.013817 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:54.013817 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:54.013817 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:21:54.013817 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:21:54.013817 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:21:54.013817 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:21:54.019948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:54.019948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:54.019948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:21:54.019948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:54.019948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:21:54.019948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:54.019948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:54.019948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:54.019948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:54.019948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:54.019948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:54.019948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:54.019948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:54.019948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:54.019948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:54.019948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:54.019948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:54.019948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:54.021025 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:54.021025 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:54.021025 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:21:54.021025 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:21:54.021025 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:21:54.021025 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:21:54.021025 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:21:54.021025 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:21:54.021025 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 12:21:54.021025 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:54.021831 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae159864-9b6d-419d-93bf-1b62058676c0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:54.021928 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 535/1068] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Jun 29 12:21:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:54.021928 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be7ed90-dc98-44a2-8347-5e1bdf0b3279 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:54.045142 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:54.061757 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f5dd74-2677-4351-af92-71dc3d82c1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:54.062590 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5dd74-2677-4351-af92-71dc3d82c1fe None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:54.062883 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5dd74-2677-4351-af92-71dc3d82c1fe None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:54.063164 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f5dd74-2677-4351-af92-71dc3d82c1fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:54.074050 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f5dd74-2677-4351-af92-71dc3d82c1fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:21:54.081747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e6f5dd74-2677-4351-af92-71dc3d82c1fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:21:54.081747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:54.081747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:54.081747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:54.081747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:54.081747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:54.081747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:54.081747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:54.081747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:54.081747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:54.081747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:54.081747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:54.081747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:21:54.081747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:21:54.081747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:21:54.081747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:21:54.082727 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:21:54.082727 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:21:54.082727 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:21:54.082727 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:21:54.082727 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:54.082727 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:54.082727 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:54.082727 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:54.082727 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:54.082727 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:54.082727 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:54.082727 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:54.082727 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:54.082727 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:54.082727 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:54.082727 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:54.083640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:54.083640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:54.083640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:54.083640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:54.083640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:21:54.083640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:21:54.083640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:21:54.083640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:21:54.083640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:21:54.083640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:21:54.083640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:21:54.083640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:54.083640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 534/1069] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:21:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:54.084654 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abc0c4-615c-4b31-95be-1e3e1dbc2891 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:54.084654 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abc0c4-615c-4b31-95be-1e3e1dbc2891 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:54.084654 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abc0c4-615c-4b31-95be-1e3e1dbc2891 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:54.086067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 535/1070] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:54.088942 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abc0c4-615c-4b31-95be-1e3e1dbc2891 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:54.129631 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be7ed90-dc98-44a2-8347-5e1bdf0b3279 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:54.162628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e59aa-a88b-40d5-922d-71a038a45647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:54.163295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e59aa-a88b-40d5-922d-71a038a45647 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:54.163580 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e59aa-a88b-40d5-922d-71a038a45647 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:54.163863 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e59aa-a88b-40d5-922d-71a038a45647 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:54.165243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e59aa-a88b-40d5-922d-71a038a45647 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:54.165557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e59aa-a88b-40d5-922d-71a038a45647 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:54.167034 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e59aa-a88b-40d5-922d-71a038a45647 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:54.184046 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 536/1071] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Jun 29 12:21:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:54.185260 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9732297-e663-451c-80fa-9e6b572c2985 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:54.185546 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9732297-e663-451c-80fa-9e6b572c2985 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:54.185832 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9732297-e663-451c-80fa-9e6b572c2985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:54.189613 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abc0c4-615c-4b31-95be-1e3e1dbc2891 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:54.197890 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:54.201136 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d9732297-e663-451c-80fa-9e6b572c2985 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:54.202356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d9732297-e663-451c-80fa-9e6b572c2985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:54.225221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfac525-1e8d-4283-95b2-a5ccf4d35d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:54.225889 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfac525-1e8d-4283-95b2-a5ccf4d35d72 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:54.226178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfac525-1e8d-4283-95b2-a5ccf4d35d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:54.226546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfac525-1e8d-4283-95b2-a5ccf4d35d72 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:54.227846 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfac525-1e8d-4283-95b2-a5ccf4d35d72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:54.228217 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfac525-1e8d-4283-95b2-a5ccf4d35d72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:54.229519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfac525-1e8d-4283-95b2-a5ccf4d35d72 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:54.242308 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 536/1072] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:21:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:54.242308 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfac525-1e8d-4283-95b2-a5ccf4d35d72 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:54.275060 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 537/1073] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:54.275899 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9732297-e663-451c-80fa-9e6b572c2985 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:54.286541 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:54.349181 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfac525-1e8d-4283-95b2-a5ccf4d35d72 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:54.380762 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9732297-e663-451c-80fa-9e6b572c2985 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:54.383915 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034ce1a-a659-4c8e-a610-b1ea59a41904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:54.384705 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034ce1a-a659-4c8e-a610-b1ea59a41904 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:54.384988 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034ce1a-a659-4c8e-a610-b1ea59a41904 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:54.385262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034ce1a-a659-4c8e-a610-b1ea59a41904 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:54.395891 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d034ce1a-a659-4c8e-a610-b1ea59a41904 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:54.397586 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d034ce1a-a659-4c8e-a610-b1ea59a41904 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:54.413675 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37553a3a-55e3-4414-ac69-b5c8f4b67893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:54.414608 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37553a3a-55e3-4414-ac69-b5c8f4b67893 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:54.414903 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37553a3a-55e3-4414-ac69-b5c8f4b67893 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:54.415174 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37553a3a-55e3-4414-ac69-b5c8f4b67893 None admin] PATH_INFO: `/v3/users/ad699d057c6c43a4a7267007208ce0a4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:54.415278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d034ce1a-a659-4c8e-a610-b1ea59a41904 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:54.415278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:54.415278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:54.415278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:54.415278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:54.415278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:54.415278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:54.415278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:54.415278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:54.415278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:54.415278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:54.415278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:54.415278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:54.415278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:54.415278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:54.416200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:54.416200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:54.416200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:54.416200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:54.416200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:54.416200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:54.416200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:54.416200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:54.416200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:54.416200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:54.416200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:54.416200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:54.416200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:54.416200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:54.416200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:54.416200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:54.416200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:54.417380 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:54.417380 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:54.417380 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:54.417380 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:54.417801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37553a3a-55e3-4414-ac69-b5c8f4b67893 None admin] RBAC: Authorizing `identity:get_user(user_id=ad699d057c6c43a4a7267007208ce0a4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:54.417898 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 537/1074] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:54.417898 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034ce1a-a659-4c8e-a610-b1ea59a41904 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:54.418053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37553a3a-55e3-4414-ac69-b5c8f4b67893 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:54.419711 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 538/1075] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:21:54 2022] GET /identity/v3/users/ad699d057c6c43a4a7267007208ce0a4 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:54.422076 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37553a3a-55e3-4414-ac69-b5c8f4b67893 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:54.443380 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:54.500447 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:54.520321 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034ce1a-a659-4c8e-a610-b1ea59a41904 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:54.524267 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37553a3a-55e3-4414-ac69-b5c8f4b67893 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:54.553246 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d377ee-b719-42f5-9879-7d93ade0648e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:54.553898 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d377ee-b719-42f5-9879-7d93ade0648e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:54.554182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d377ee-b719-42f5-9879-7d93ade0648e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:54.554487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d377ee-b719-42f5-9879-7d93ade0648e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:54.557416 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35534f2-f9d3-4c32-a983-b27a5c2d7ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:54.558066 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35534f2-f9d3-4c32-a983-b27a5c2d7ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:54.558352 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35534f2-f9d3-4c32-a983-b27a5c2d7ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:54.558662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35534f2-f9d3-4c32-a983-b27a5c2d7ad5 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:54.559384 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35534f2-f9d3-4c32-a983-b27a5c2d7ad5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:54.559697 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35534f2-f9d3-4c32-a983-b27a5c2d7ad5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:54.561066 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35534f2-f9d3-4c32-a983-b27a5c2d7ad5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:54.573238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 539/1076] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:54.574027 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35534f2-f9d3-4c32-a983-b27a5c2d7ad5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:54.627782 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d377ee-b719-42f5-9879-7d93ade0648e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:54.629154 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d377ee-b719-42f5-9879-7d93ade0648e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:54.650471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:54.676108 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35534f2-f9d3-4c32-a983-b27a5c2d7ad5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:54.709625 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562cc37-f595-45ef-b527-67b94804c041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:54.710495 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562cc37-f595-45ef-b527-67b94804c041 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:54.710777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562cc37-f595-45ef-b527-67b94804c041 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:54.711052 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4562cc37-f595-45ef-b527-67b94804c041 None admin] PATH_INFO: `/v3/projects/9f8f17b758264cf38b4c2b5fe48194da` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:54.712472 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4562cc37-f595-45ef-b527-67b94804c041 None admin] RBAC: Authorizing `identity:get_project(project_id=9f8f17b758264cf38b4c2b5fe48194da)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:54.713890 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4562cc37-f595-45ef-b527-67b94804c041 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:54.715763 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 540/1077] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:21:54 2022] GET /identity/v3/projects/9f8f17b758264cf38b4c2b5fe48194da => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:54.716352 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562cc37-f595-45ef-b527-67b94804c041 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:54.719584 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:54.738817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 538/1078] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:21:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 12:21:54.738817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d377ee-b719-42f5-9879-7d93ade0648e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:54.819224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4562cc37-f595-45ef-b527-67b94804c041 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:54.831982 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d377ee-b719-42f5-9879-7d93ade0648e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:54.851304 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:54.853616 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c625694-74b1-4c95-bb35-075f7a59790b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:54.854662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c625694-74b1-4c95-bb35-075f7a59790b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:54.854950 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c625694-74b1-4c95-bb35-075f7a59790b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:54.855231 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c625694-74b1-4c95-bb35-075f7a59790b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:54.862040 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c625694-74b1-4c95-bb35-075f7a59790b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:54.863532 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c625694-74b1-4c95-bb35-075f7a59790b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:54.865777 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028f233-dd7b-463b-b07d-da93a18139cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:54.866688 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028f233-dd7b-463b-b07d-da93a18139cb None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:54.866974 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028f233-dd7b-463b-b07d-da93a18139cb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:54.867252 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6028f233-dd7b-463b-b07d-da93a18139cb None admin] PATH_INFO: `/v3/projects/9f8f17b758264cf38b4c2b5fe48194da/users/ad699d057c6c43a4a7267007208ce0a4/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:54.869840 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6028f233-dd7b-463b-b07d-da93a18139cb None admin] RBAC: Authorizing `identity:create_grant(project_id=9f8f17b758264cf38b4c2b5fe48194da, user_id=ad699d057c6c43a4a7267007208ce0a4, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:54.871207 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6028f233-dd7b-463b-b07d-da93a18139cb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:54.879210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6c625694-74b1-4c95-bb35-075f7a59790b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:54.879210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:54.879210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:54.879210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:54.879210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:54.879210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:54.879210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:54.879210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:54.879210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:54.879210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:54.879210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:54.879210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:54.879210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:21:54.879210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:21:54.879210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:21:54.880111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:21:54.880111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:54.880111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:54.880111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:54.880111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:54.880111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:54.880111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:54.880111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:54.880111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:54.880111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:54.880111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:54.880111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:54.880111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:54.880111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:54.880111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:54.880111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:54.880111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:54.881752 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:21:54.881752 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:21:54.881752 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:21:54.881752 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:21:54.881752 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:21:54.881752 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:21:54.881752 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:54.881752 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:54.881752 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 541/1079] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:21:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:54.881752 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c625694-74b1-4c95-bb35-075f7a59790b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:54.884997 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 539/1080] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 12:21:54 2022] PUT /identity/v3/projects/9f8f17b758264cf38b4c2b5fe48194da/users/ad699d057c6c43a4a7267007208ce0a4/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:21:54.886148 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028f233-dd7b-463b-b07d-da93a18139cb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:54.956561 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:55.000494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c625694-74b1-4c95-bb35-075f7a59790b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:55.010796 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6028f233-dd7b-463b-b07d-da93a18139cb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:55.047342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c664864-b2eb-4afa-950f-23dc1948617d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:55.048146 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c664864-b2eb-4afa-950f-23dc1948617d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:55.048435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c664864-b2eb-4afa-950f-23dc1948617d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:55.048715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c664864-b2eb-4afa-950f-23dc1948617d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:55.052989 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e68a0-1383-4881-846c-16f4d0709d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:55.053661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e68a0-1383-4881-846c-16f4d0709d12 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:55.053949 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e68a0-1383-4881-846c-16f4d0709d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:55.054229 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e68a0-1383-4881-846c-16f4d0709d12 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:55.055053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e68a0-1383-4881-846c-16f4d0709d12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:55.055912 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e68a0-1383-4881-846c-16f4d0709d12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:55.057353 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e68a0-1383-4881-846c-16f4d0709d12 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:55.062132 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c664864-b2eb-4afa-950f-23dc1948617d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:55.063603 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c664864-b2eb-4afa-950f-23dc1948617d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:55.070316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 540/1081] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:21:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:55.076176 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e68a0-1383-4881-846c-16f4d0709d12 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:55.083158 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-3c664864-b2eb-4afa-950f-23dc1948617d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:55.083158 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:55.083158 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:55.083158 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:55.083158 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:55.083158 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:55.083158 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:55.083158 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:55.083158 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:55.083158 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:55.083158 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:55.083158 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:55.083158 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:55.083158 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:55.083158 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:55.084053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:55.084053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:55.084053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:55.084053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:55.084053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:55.084053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:55.084053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:55.084053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:55.084053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:55.084053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:55.084053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:55.084053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:55.084053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:55.084053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:55.084053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:55.084053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:55.084053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:55.085092 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:55.085092 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:55.085092 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:55.085092 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:55.085092 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 542/1082] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:55.091328 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c664864-b2eb-4afa-950f-23dc1948617d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:55.122635 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:55.187098 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e68a0-1383-4881-846c-16f4d0709d12 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:55.198828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c664864-b2eb-4afa-950f-23dc1948617d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:55.216868 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:55.224273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb190b-05c3-4993-b6d1-7953ad2a21a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:55.225189 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb190b-05c3-4993-b6d1-7953ad2a21a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:55.225457 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb190b-05c3-4993-b6d1-7953ad2a21a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:55.225736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb190b-05c3-4993-b6d1-7953ad2a21a4 None admin] PATH_INFO: `/v3/projects/f043ade9e4f748d4adc010ffb0a47a46/users/3ac7391a3c41499d94e5f97478797313/roles/7dedcc77e86a4fee94bf84105c0ffd71` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:55.228849 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb190b-05c3-4993-b6d1-7953ad2a21a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f043ade9e4f748d4adc010ffb0a47a46, user_id=3ac7391a3c41499d94e5f97478797313, role_id=7dedcc77e86a4fee94bf84105c0ffd71)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:55.230281 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb190b-05c3-4993-b6d1-7953ad2a21a4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:55.237877 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4139a7f-46c3-4a10-ad52-b117971a10f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:55.238597 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4139a7f-46c3-4a10-ad52-b117971a10f7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:55.238865 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4139a7f-46c3-4a10-ad52-b117971a10f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:55.239142 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4139a7f-46c3-4a10-ad52-b117971a10f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:55.239884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4139a7f-46c3-4a10-ad52-b117971a10f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:55.240188 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4139a7f-46c3-4a10-ad52-b117971a10f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:55.241558 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4139a7f-46c3-4a10-ad52-b117971a10f7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:55.246796 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 541/1083] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 12:21:55 2022] PUT /identity/v3/projects/f043ade9e4f748d4adc010ffb0a47a46/users/3ac7391a3c41499d94e5f97478797313/roles/7dedcc77e86a4fee94bf84105c0ffd71 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:21:55.260524 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 543/1084] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:56.935052 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06440353-ce05-441d-a75e-2408ef2628b9 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:56.935584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06440353-ce05-441d-a75e-2408ef2628b9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:56.935678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06440353-ce05-441d-a75e-2408ef2628b9 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:56.936964 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 542/1085] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:21:56.942125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddcbf6-a8b0-4d1f-9432-1ceff335f6af None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:56.942560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddcbf6-a8b0-4d1f-9432-1ceff335f6af None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:56.942725 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddcbf6-a8b0-4d1f-9432-1ceff335f6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:56.958798 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-fbddcbf6-a8b0-4d1f-9432-1ceff335f6af None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:56.959973 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fbddcbf6-a8b0-4d1f-9432-1ceff335f6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:57.026189 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 544/1086] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:21:57.049910 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06440353-ce05-441d-a75e-2408ef2628b9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:57.147639 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:57.152749 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06440353-ce05-441d-a75e-2408ef2628b9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:57.186620 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ed662-be85-4478-8e68-67761fe0ae1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:57.187326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ed662-be85-4478-8e68-67761fe0ae1f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:57.187610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ed662-be85-4478-8e68-67761fe0ae1f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:57.187884 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ed662-be85-4478-8e68-67761fe0ae1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:57.198318 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ed662-be85-4478-8e68-67761fe0ae1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:21:57.205832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d99ed662-be85-4478-8e68-67761fe0ae1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:21:57.205832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:57.205832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:57.205832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:57.205832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:57.205832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:57.205832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:57.205832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:57.205832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:57.205832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:57.205832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:57.205832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:57.205832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:21:57.205832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:21:57.205832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:21:57.205832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:21:57.206813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:21:57.206813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:21:57.206813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:21:57.206813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:21:57.206813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:57.206813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:57.206813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:57.206813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:57.206813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:57.206813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:57.206813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:57.206813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:57.206813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:57.206813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:57.206813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:57.206813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:57.207697 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:57.207697 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:57.207697 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:57.207697 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:57.207697 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:21:57.207697 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:21:57.207697 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:21:57.207697 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:21:57.207697 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:21:57.207697 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:21:57.207697 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:21:57.207697 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:57.207697 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 543/1087] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:21:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:57.212978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbddcbf6-a8b0-4d1f-9432-1ceff335f6af None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:57.310526 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbddcbf6-a8b0-4d1f-9432-1ceff335f6af None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:57.310728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172b018-12b1-4290-a212-61f77e420d15 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:57.311001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172b018-12b1-4290-a212-61f77e420d15 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:57.311283 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172b018-12b1-4290-a212-61f77e420d15 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:57.312649 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 544/1088] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:21:57.317389 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8539d6e-a6f7-4efa-befe-8ccd5bfc996f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:57.317679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8539d6e-a6f7-4efa-befe-8ccd5bfc996f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:57.317814 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:57.317963 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8539d6e-a6f7-4efa-befe-8ccd5bfc996f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:57.335489 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c8539d6e-a6f7-4efa-befe-8ccd5bfc996f None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:21:57.336733 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c8539d6e-a6f7-4efa-befe-8ccd5bfc996f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:21:57.345523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e812af-1e6e-4506-9411-97cf6ffda33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:57.346191 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e812af-1e6e-4506-9411-97cf6ffda33d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:57.346498 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e812af-1e6e-4506-9411-97cf6ffda33d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:57.346777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e812af-1e6e-4506-9411-97cf6ffda33d None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:57.347516 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e812af-1e6e-4506-9411-97cf6ffda33d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:57.347821 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e812af-1e6e-4506-9411-97cf6ffda33d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:57.349184 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e812af-1e6e-4506-9411-97cf6ffda33d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:57.362485 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 545/1089] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:21:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:57.363368 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e812af-1e6e-4506-9411-97cf6ffda33d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:57.408019 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 545/1090] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:21:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 12:21:57.432230 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8539d6e-a6f7-4efa-befe-8ccd5bfc996f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:57.463745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e812af-1e6e-4506-9411-97cf6ffda33d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:57.497012 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d2dad-cb1b-44a8-9570-50387870a3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:57.497878 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d2dad-cb1b-44a8-9570-50387870a3dd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:57.498159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d2dad-cb1b-44a8-9570-50387870a3dd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:57.498449 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d2dad-cb1b-44a8-9570-50387870a3dd None admin] PATH_INFO: `/v3/users/ad699d057c6c43a4a7267007208ce0a4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:57.499659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d2dad-cb1b-44a8-9570-50387870a3dd None admin] RBAC: Authorizing `identity:get_user(user_id=ad699d057c6c43a4a7267007208ce0a4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:57.501111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d2dad-cb1b-44a8-9570-50387870a3dd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:57.502778 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 546/1091] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:21:57 2022] GET /identity/v3/users/ad699d057c6c43a4a7267007208ce0a4 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:57.508840 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d2dad-cb1b-44a8-9570-50387870a3dd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:57.522113 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:57.532118 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8539d6e-a6f7-4efa-befe-8ccd5bfc996f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:57.543434 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:57.568296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a83be5-af87-4014-930f-b4d9f7c26b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:57.569128 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a83be5-af87-4014-930f-b4d9f7c26b7f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:57.569412 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a83be5-af87-4014-930f-b4d9f7c26b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:57.569691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a83be5-af87-4014-930f-b4d9f7c26b7f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:57.580342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a83be5-af87-4014-930f-b4d9f7c26b7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:21:57.588518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a4a83be5-af87-4014-930f-b4d9f7c26b7f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 12:21:57.588518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:57.588518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:57.588518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:57.588518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:57.588518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:57.588518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:57.588518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:57.588518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:57.588518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:57.588518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:57.588518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:57.588518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:21:57.588518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:21:57.588518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:21:57.588518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:21:57.589505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:21:57.589505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:21:57.589505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:21:57.589505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:21:57.589505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:57.589505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:57.589505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:57.589505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:57.589505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:57.589505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:57.589505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:57.589505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:57.589505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:57.589505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:57.589505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:57.589505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:57.595583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:57.595583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:57.595583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:57.595583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:57.595583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:21:57.595583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:21:57.595583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:21:57.595583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:21:57.595583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:21:57.595583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:21:57.595583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 29 12:21:57.595583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:57.595583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 546/1092] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Jun 29 12:21:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:57.596275 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a83be5-af87-4014-930f-b4d9f7c26b7f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:57.606739 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d2dad-cb1b-44a8-9570-50387870a3dd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:57.640389 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37df3c9-e057-43fc-bbd5-ba1f23703cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:57.641031 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37df3c9-e057-43fc-bbd5-ba1f23703cc5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:57.641310 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37df3c9-e057-43fc-bbd5-ba1f23703cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:57.641591 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37df3c9-e057-43fc-bbd5-ba1f23703cc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:57.694821 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a83be5-af87-4014-930f-b4d9f7c26b7f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:57.707346 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37df3c9-e057-43fc-bbd5-ba1f23703cc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:57.708811 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37df3c9-e057-43fc-bbd5-ba1f23703cc5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:57.729045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45e77f-848c-46fe-a579-0ea6da65f369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:57.729775 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45e77f-848c-46fe-a579-0ea6da65f369 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:57.730073 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45e77f-848c-46fe-a579-0ea6da65f369 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:57.730357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e45e77f-848c-46fe-a579-0ea6da65f369 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:57.731136 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45e77f-848c-46fe-a579-0ea6da65f369 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:57.731451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45e77f-848c-46fe-a579-0ea6da65f369 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:57.732816 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e45e77f-848c-46fe-a579-0ea6da65f369 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:57.740762 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 547/1093] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Jun 29 12:21:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:57.743393 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:57.747374 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45e77f-848c-46fe-a579-0ea6da65f369 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:57.762404 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:57.808637 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 547/1094] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:21:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 29 12:21:57.814305 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37df3c9-e057-43fc-bbd5-ba1f23703cc5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:57.845780 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e45e77f-848c-46fe-a579-0ea6da65f369 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:57.878584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac3f7a1-fc0d-4ea1-9847-8a477d54b00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:57.879375 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3f7a1-fc0d-4ea1-9847-8a477d54b00e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:57.879664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3f7a1-fc0d-4ea1-9847-8a477d54b00e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:57.879941 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3f7a1-fc0d-4ea1-9847-8a477d54b00e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:57.891711 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac3f7a1-fc0d-4ea1-9847-8a477d54b00e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:57.893148 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac3f7a1-fc0d-4ea1-9847-8a477d54b00e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:57.911708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9ac3f7a1-fc0d-4ea1-9847-8a477d54b00e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:57.911708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:57.911708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:57.911708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:57.911708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:57.911708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:57.911708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:57.911708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:57.911708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:57.911708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:57.911708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:57.911708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:57.911708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:57.911708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:57.911708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:57.923310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:57.923310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:57.923310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:57.923310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:57.923310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:57.923310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:57.923310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:57.923310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:57.923310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:57.923310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:57.923310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:57.923310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:57.923310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:57.923310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:57.923310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:57.923310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:57.923310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:57.924595 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:57.924595 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:57.924595 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:57.924595 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:57.924877 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37df3c9-e057-43fc-bbd5-ba1f23703cc5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:57.924982 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 548/1095] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:57.924982 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac3f7a1-fc0d-4ea1-9847-8a477d54b00e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:57.953322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ced1d-0afa-4305-8642-e3ed9f9029b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:57.954185 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ced1d-0afa-4305-8642-e3ed9f9029b4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:57.954503 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ced1d-0afa-4305-8642-e3ed9f9029b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:57.954787 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ced1d-0afa-4305-8642-e3ed9f9029b4 None admin] PATH_INFO: `/v3/projects/9f8f17b758264cf38b4c2b5fe48194da` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:57.955964 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47ced1d-0afa-4305-8642-e3ed9f9029b4 None admin] RBAC: Authorizing `identity:get_project(project_id=9f8f17b758264cf38b4c2b5fe48194da)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:57.957403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47ced1d-0afa-4305-8642-e3ed9f9029b4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:57.959137 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 548/1096] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:21:57 2022] GET /identity/v3/projects/9f8f17b758264cf38b4c2b5fe48194da => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:57.965797 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ced1d-0afa-4305-8642-e3ed9f9029b4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:57.969162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:57.987374 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:58.027160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac3f7a1-fc0d-4ea1-9847-8a477d54b00e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:58.060123 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8385698-18b4-4d99-bca6-f44ee77918f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:58.060788 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8385698-18b4-4d99-bca6-f44ee77918f6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:58.061075 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8385698-18b4-4d99-bca6-f44ee77918f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:58.061356 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8385698-18b4-4d99-bca6-f44ee77918f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:58.062077 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8385698-18b4-4d99-bca6-f44ee77918f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:58.062389 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8385698-18b4-4d99-bca6-f44ee77918f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:58.063340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ced1d-0afa-4305-8642-e3ed9f9029b4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:58.063798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8385698-18b4-4d99-bca6-f44ee77918f6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:58.076222 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 549/1097] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:58.081868 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8385698-18b4-4d99-bca6-f44ee77918f6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:58.097450 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b4804d-0c68-43d2-9129-d3453d185c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:58.098102 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4804d-0c68-43d2-9129-d3453d185c4f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:58.098391 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4804d-0c68-43d2-9129-d3453d185c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:58.098708 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4804d-0c68-43d2-9129-d3453d185c4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:58.099533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b4804d-0c68-43d2-9129-d3453d185c4f None admin] RBAC: Adding query filter params (role.id=3a7f4bf556344407a6d9540c89d13641, scope.project.id=9f8f17b758264cf38b4c2b5fe48194da, user.id=ad699d057c6c43a4a7267007208ce0a4) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:58.099849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b4804d-0c68-43d2-9129-d3453d185c4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:58.101276 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b4804d-0c68-43d2-9129-d3453d185c4f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:58.122468 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 549/1098] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:21:57 2022] GET /identity/v3/role_assignments?user.id=ad699d057c6c43a4a7267007208ce0a4&scope.project.id=9f8f17b758264cf38b4c2b5fe48194da&role.id=3a7f4bf556344407a6d9540c89d13641 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:21:58.186095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8385698-18b4-4d99-bca6-f44ee77918f6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:58.211013 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:58.223034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866a0f2-e8b4-4191-95b7-90d51f55d171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:58.223845 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866a0f2-e8b4-4191-95b7-90d51f55d171 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:58.224131 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866a0f2-e8b4-4191-95b7-90d51f55d171 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:58.224410 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866a0f2-e8b4-4191-95b7-90d51f55d171 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:58.232883 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9866a0f2-e8b4-4191-95b7-90d51f55d171 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:58.234360 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9866a0f2-e8b4-4191-95b7-90d51f55d171 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:58.249608 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9866a0f2-e8b4-4191-95b7-90d51f55d171 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 29 12:21:58.249608 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:58.249608 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:58.249608 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:58.249608 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:58.249608 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:58.249608 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:58.249608 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:58.249608 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:58.249608 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:58.249608 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:58.249608 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:58.249608 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:21:58.249608 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:21:58.249608 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:21:58.249608 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:21:58.250716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:58.250716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:58.250716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:21:58.250716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:58.250716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:21:58.250716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:21:58.250716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:58.250716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:58.250716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:58.250716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:58.250716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:58.250716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:58.250716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:58.250716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:58.250716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:58.250716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:58.250716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:58.250716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:58.251700 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:58.251700 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:58.251700 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:21:58.251700 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:21:58.251700 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:21:58.251700 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:21:58.251700 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:21:58.251700 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:21:58.251700 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 29 12:21:58.251700 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:21:58.251700 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 550/1099] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Jun 29 12:21:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:21:58.256801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b4804d-0c68-43d2-9129-d3453d185c4f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:58.324917 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:58.359068 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b4804d-0c68-43d2-9129-d3453d185c4f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:58.393207 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7ee0d-4d91-41ef-ae0b-a0f7f63e211a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:58.393869 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7ee0d-4d91-41ef-ae0b-a0f7f63e211a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:58.394442 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7ee0d-4d91-41ef-ae0b-a0f7f63e211a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:58.394722 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7ee0d-4d91-41ef-ae0b-a0f7f63e211a None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:58.395555 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7ee0d-4d91-41ef-ae0b-a0f7f63e211a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:58.395865 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7ee0d-4d91-41ef-ae0b-a0f7f63e211a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:58.397544 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7ee0d-4d91-41ef-ae0b-a0f7f63e211a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:58.415348 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 550/1100] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Jun 29 12:21:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:58.420949 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866a0f2-e8b4-4191-95b7-90d51f55d171 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:58.519815 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866a0f2-e8b4-4191-95b7-90d51f55d171 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:58.553116 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e4948-22a7-4217-bad2-2b03eb612af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:58.553759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e4948-22a7-4217-bad2-2b03eb612af4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:58.554042 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e4948-22a7-4217-bad2-2b03eb612af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:58.554317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e4948-22a7-4217-bad2-2b03eb612af4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:58.606091 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:58.621974 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e4948-22a7-4217-bad2-2b03eb612af4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:58.623448 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e4948-22a7-4217-bad2-2b03eb612af4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:58.719595 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 551/1101] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:21:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 12:21:58.725299 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7ee0d-4d91-41ef-ae0b-a0f7f63e211a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:58.823440 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7ee0d-4d91-41ef-ae0b-a0f7f63e211a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:58.847931 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:58.857030 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182791d-a1bd-4c09-823b-eeff11c62c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:58.857822 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182791d-a1bd-4c09-823b-eeff11c62c5c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:58.858108 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182791d-a1bd-4c09-823b-eeff11c62c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:58.858389 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182791d-a1bd-4c09-823b-eeff11c62c5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:58.868844 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d182791d-a1bd-4c09-823b-eeff11c62c5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:58.870282 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d182791d-a1bd-4c09-823b-eeff11c62c5c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:58.887826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d182791d-a1bd-4c09-823b-eeff11c62c5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:58.887826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:58.887826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:58.887826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:58.887826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:58.887826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:58.887826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:58.887826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:58.887826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:58.887826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:58.887826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:58.887826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:58.887826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:21:58.887826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:21:58.887826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:21:58.888871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:21:58.888871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:58.888871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:58.888871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:58.888871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:58.888871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:58.888871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:58.888871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:58.888871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:58.888871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:58.888871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:58.888871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:58.888871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:58.888871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:58.888871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:58.888871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:58.888871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:58.889770 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:21:58.889770 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:21:58.889770 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:21:58.889770 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:58.889770 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 551/1102] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:21:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:58.894653 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e4948-22a7-4217-bad2-2b03eb612af4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:58.975950 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:58.992290 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e4948-22a7-4217-bad2-2b03eb612af4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:59.026362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a015dd-7619-417e-a6b4-bf0c4e46854f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:59.027049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a015dd-7619-417e-a6b4-bf0c4e46854f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:59.027339 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a015dd-7619-417e-a6b4-bf0c4e46854f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:59.027621 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a015dd-7619-417e-a6b4-bf0c4e46854f None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:59.028339 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a015dd-7619-417e-a6b4-bf0c4e46854f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:59.028653 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a015dd-7619-417e-a6b4-bf0c4e46854f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:59.030029 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a015dd-7619-417e-a6b4-bf0c4e46854f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:59.041663 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 552/1103] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:21:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:59.047117 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182791d-a1bd-4c09-823b-eeff11c62c5c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:59.144223 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182791d-a1bd-4c09-823b-eeff11c62c5c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:59.177026 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a138f-a066-4e26-9165-4cd2c0321212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:59.177789 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a138f-a066-4e26-9165-4cd2c0321212 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:59.178073 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a138f-a066-4e26-9165-4cd2c0321212 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:59.178355 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507a138f-a066-4e26-9165-4cd2c0321212 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:59.185376 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a138f-a066-4e26-9165-4cd2c0321212 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:59.186850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507a138f-a066-4e26-9165-4cd2c0321212 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:59.200897 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-507a138f-a066-4e26-9165-4cd2c0321212 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:59.200897 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:21:59.200897 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:21:59.200897 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:21:59.200897 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:21:59.200897 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:21:59.200897 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:21:59.200897 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:21:59.200897 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:21:59.200897 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:21:59.200897 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:21:59.200897 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:21:59.200897 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:21:59.200897 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:21:59.200897 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:21:59.201994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:21:59.201994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:21:59.201994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:21:59.201994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:21:59.201994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:21:59.201994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:21:59.201994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:21:59.201994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:21:59.201994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:21:59.201994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:21:59.201994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:21:59.201994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:21:59.201994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:21:59.201994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:21:59.201994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:21:59.201994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:21:59.201994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:21:59.202973 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:21:59.202973 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:21:59.202973 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:21:59.202973 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:21:59.202973 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:21:59.202973 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:21:59.202973 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:21:59.202973 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:21:59.202973 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 552/1104] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:21:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:21:59.207872 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a015dd-7619-417e-a6b4-bf0c4e46854f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:59.315266 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a015dd-7619-417e-a6b4-bf0c4e46854f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:59.349622 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c129e-6c68-4470-8cab-ce15ef40176b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:59.350282 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c129e-6c68-4470-8cab-ce15ef40176b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:59.350600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c129e-6c68-4470-8cab-ce15ef40176b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:59.350880 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c129e-6c68-4470-8cab-ce15ef40176b None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:59.351665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c129e-6c68-4470-8cab-ce15ef40176b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:59.352538 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c129e-6c68-4470-8cab-ce15ef40176b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:59.354583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c129e-6c68-4470-8cab-ce15ef40176b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:59.366675 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 553/1105] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:21:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:21:59.373195 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a138f-a066-4e26-9165-4cd2c0321212 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:21:59.412052 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:21:59.472054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507a138f-a066-4e26-9165-4cd2c0321212 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:21:59.505027 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8bf28-d894-4af7-ac9a-0027d333619f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:21:59.505669 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8bf28-d894-4af7-ac9a-0027d333619f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:21:59.505955 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8bf28-d894-4af7-ac9a-0027d333619f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:21:59.506236 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8bf28-d894-4af7-ac9a-0027d333619f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:21:59.507123 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c8bf28-d894-4af7-ac9a-0027d333619f None admin] RBAC: Adding query filter params (role.id=7dedcc77e86a4fee94bf84105c0ffd71, scope.project.id=f043ade9e4f748d4adc010ffb0a47a46, user.id=3ac7391a3c41499d94e5f97478797313) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:21:59.507437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c8bf28-d894-4af7-ac9a-0027d333619f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:21:59.508989 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c8bf28-d894-4af7-ac9a-0027d333619f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:21:59.514759 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 553/1106] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:21:59 2022] GET /identity/v3/role_assignments?user.id=3ac7391a3c41499d94e5f97478797313&scope.project.id=f043ade9e4f748d4adc010ffb0a47a46&role.id=7dedcc77e86a4fee94bf84105c0ffd71 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:00.182920 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2a1f2-3f46-4a72-97fa-bbc0f0158832 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:00.183545 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2a1f2-3f46-4a72-97fa-bbc0f0158832 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:00.183545 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2a1f2-3f46-4a72-97fa-bbc0f0158832 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:00.184848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 554/1107] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:22:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:22:00.190214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3d44f-09a0-43b0-a094-d908c5b2be5b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:00.190736 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3d44f-09a0-43b0-a094-d908c5b2be5b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:00.190866 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3d44f-09a0-43b0-a094-d908c5b2be5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:00.207679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3ce3d44f-09a0-43b0-a094-d908c5b2be5b None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:22:00.208858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3ce3d44f-09a0-43b0-a094-d908c5b2be5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:22:00.278696 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 554/1108] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:22:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:22:00.302844 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2a1f2-3f46-4a72-97fa-bbc0f0158832 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:00.307363 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:00.407494 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2a1f2-3f46-4a72-97fa-bbc0f0158832 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:00.440597 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b085fb0-50ae-4967-be07-7f6efcaa94c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:00.441306 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b085fb0-50ae-4967-be07-7f6efcaa94c7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:00.441590 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b085fb0-50ae-4967-be07-7f6efcaa94c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:00.441866 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b085fb0-50ae-4967-be07-7f6efcaa94c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:00.443034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b085fb0-50ae-4967-be07-7f6efcaa94c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:00.444456 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b085fb0-50ae-4967-be07-7f6efcaa94c7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:00.446800 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 555/1109] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:22:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:00.452504 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3d44f-09a0-43b0-a094-d908c5b2be5b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:00.481756 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:00.548297 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3d44f-09a0-43b0-a094-d908c5b2be5b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:00.581390 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02cdc79-edb7-4624-bd29-59b1b9636776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:00.582047 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02cdc79-edb7-4624-bd29-59b1b9636776 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:00.582331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02cdc79-edb7-4624-bd29-59b1b9636776 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:00.582660 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02cdc79-edb7-4624-bd29-59b1b9636776 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:00.583375 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02cdc79-edb7-4624-bd29-59b1b9636776 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:00.584798 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02cdc79-edb7-4624-bd29-59b1b9636776 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:00.604000 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 555/1110] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 12:22:00 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:22:02.643168 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b414e-b50b-475a-bd98-0b0f1a12d662 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:02.643723 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b414e-b50b-475a-bd98-0b0f1a12d662 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:02.643723 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b414e-b50b-475a-bd98-0b0f1a12d662 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:02.644981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 556/1111] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:22:02.650994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3a324-dda0-4c7d-9e13-cbfd7a553045 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:02.651402 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3a324-dda0-4c7d-9e13-cbfd7a553045 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:02.651554 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd3a324-dda0-4c7d-9e13-cbfd7a553045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:02.669187 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1dd3a324-dda0-4c7d-9e13-cbfd7a553045 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:22:02.670350 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1dd3a324-dda0-4c7d-9e13-cbfd7a553045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:22:02.734278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 556/1112] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:22:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:22:02.757836 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b414e-b50b-475a-bd98-0b0f1a12d662 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:02.828490 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:02.858919 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b414e-b50b-475a-bd98-0b0f1a12d662 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:02.891580 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eeaca5-af3c-4367-92f3-3f27ee62878e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:02.892420 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eeaca5-af3c-4367-92f3-3f27ee62878e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:02.892703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eeaca5-af3c-4367-92f3-3f27ee62878e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:02.892977 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eeaca5-af3c-4367-92f3-3f27ee62878e None admin] PATH_INFO: `/v3/roles/6005bd32383a4706a69108f7a2cf3ed1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:02.894485 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eeaca5-af3c-4367-92f3-3f27ee62878e None admin] RBAC: Authorizing `identity:get_role(role_id=6005bd32383a4706a69108f7a2cf3ed1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:02.895792 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eeaca5-af3c-4367-92f3-3f27ee62878e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:02.897390 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 557/1113] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:22:02 2022] GET /identity/v3/roles/6005bd32383a4706a69108f7a2cf3ed1 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:02.903648 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3a324-dda0-4c7d-9e13-cbfd7a553045 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:02.921074 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:02.999583 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd3a324-dda0-4c7d-9e13-cbfd7a553045 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:03.032917 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e8774-1d01-41ef-bc0a-13c99d4dbec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:03.033774 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e8774-1d01-41ef-bc0a-13c99d4dbec2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:03.034062 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e8774-1d01-41ef-bc0a-13c99d4dbec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:03.034344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e8774-1d01-41ef-bc0a-13c99d4dbec2 None admin] PATH_INFO: `/v3/users/a86ea826c5404521a1dc6b2162730f43` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:03.046963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e8774-1d01-41ef-bc0a-13c99d4dbec2 None admin] RBAC: Authorizing `identity:get_user(user_id=a86ea826c5404521a1dc6b2162730f43)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:03.048399 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e8774-1d01-41ef-bc0a-13c99d4dbec2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:03.050033 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 557/1114] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:22:02 2022] GET /identity/v3/users/a86ea826c5404521a1dc6b2162730f43 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:03.055691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eeaca5-af3c-4367-92f3-3f27ee62878e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:03.157204 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eeaca5-af3c-4367-92f3-3f27ee62878e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:03.189415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801552bb-0059-4c12-8ea4-fa44750474e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:03.190054 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801552bb-0059-4c12-8ea4-fa44750474e1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:03.190344 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801552bb-0059-4c12-8ea4-fa44750474e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:03.190810 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801552bb-0059-4c12-8ea4-fa44750474e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:03.201372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:03.257146 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801552bb-0059-4c12-8ea4-fa44750474e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:03.259283 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801552bb-0059-4c12-8ea4-fa44750474e1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:03.356333 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 558/1115] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:22:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 12:22:03.362147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e8774-1d01-41ef-bc0a-13c99d4dbec2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:03.436066 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:03.458561 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e8774-1d01-41ef-bc0a-13c99d4dbec2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:03.490071 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf183c-64e5-4395-a114-9c3df4d07cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:03.490935 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf183c-64e5-4395-a114-9c3df4d07cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:03.491217 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf183c-64e5-4395-a114-9c3df4d07cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:03.491494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf183c-64e5-4395-a114-9c3df4d07cc9 None admin] PATH_INFO: `/v3/projects/9f8f17b758264cf38b4c2b5fe48194da` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:03.493165 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf183c-64e5-4395-a114-9c3df4d07cc9 None admin] RBAC: Authorizing `identity:get_project(project_id=9f8f17b758264cf38b4c2b5fe48194da)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:03.494620 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf183c-64e5-4395-a114-9c3df4d07cc9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:03.496284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 558/1116] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:22:03 2022] GET /identity/v3/projects/9f8f17b758264cf38b4c2b5fe48194da => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:03.502442 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801552bb-0059-4c12-8ea4-fa44750474e1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:03.543301 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:03.599204 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801552bb-0059-4c12-8ea4-fa44750474e1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:03.631999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f8daf7-65f3-40ed-af83-a71b9b55f9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:03.632644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8daf7-65f3-40ed-af83-a71b9b55f9dc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:03.632931 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8daf7-65f3-40ed-af83-a71b9b55f9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:03.633925 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f8daf7-65f3-40ed-af83-a71b9b55f9dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:03.634874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8daf7-65f3-40ed-af83-a71b9b55f9dc None admin] RBAC: Adding query filter params (role.id=6005bd32383a4706a69108f7a2cf3ed1, scope.project.id=9f8f17b758264cf38b4c2b5fe48194da, user.id=a86ea826c5404521a1dc6b2162730f43) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:22:03.635199 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8daf7-65f3-40ed-af83-a71b9b55f9dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:03.636624 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f8daf7-65f3-40ed-af83-a71b9b55f9dc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:03.642122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 559/1117] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:22:03 2022] GET /identity/v3/role_assignments?user.id=a86ea826c5404521a1dc6b2162730f43&scope.project.id=9f8f17b758264cf38b4c2b5fe48194da&role.id=6005bd32383a4706a69108f7a2cf3ed1 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:05.657767 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ac544-df42-4706-9c40-bf5bae4fdcff None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:05.658279 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ac544-df42-4706-9c40-bf5bae4fdcff None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:05.658279 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ac544-df42-4706-9c40-bf5bae4fdcff None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:05.659649 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 559/1118] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:22:05.665068 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8da86d-a78e-4ce4-82d9-a8b237a9739d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:05.665475 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8da86d-a78e-4ce4-82d9-a8b237a9739d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:05.665624 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8da86d-a78e-4ce4-82d9-a8b237a9739d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:05.682319 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fb8da86d-a78e-4ce4-82d9-a8b237a9739d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:22:05.683521 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fb8da86d-a78e-4ce4-82d9-a8b237a9739d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:22:05.746646 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 560/1119] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:22:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:22:05.769393 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ac544-df42-4706-9c40-bf5bae4fdcff None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:05.865856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ac544-df42-4706-9c40-bf5bae4fdcff None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:05.898362 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d4d97-3121-4175-bc8c-4178487ea5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:05.899244 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d4d97-3121-4175-bc8c-4178487ea5bd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:05.899522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d4d97-3121-4175-bc8c-4178487ea5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:05.899798 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d4d97-3121-4175-bc8c-4178487ea5bd None admin] PATH_INFO: `/v3/roles/6005bd32383a4706a69108f7a2cf3ed1` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:05.901240 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d4d97-3121-4175-bc8c-4178487ea5bd None admin] RBAC: Authorizing `identity:get_role(role_id=6005bd32383a4706a69108f7a2cf3ed1)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:05.902628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d4d97-3121-4175-bc8c-4178487ea5bd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:05.903961 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 560/1120] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:22:05 2022] GET /identity/v3/roles/6005bd32383a4706a69108f7a2cf3ed1 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:05.909663 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8da86d-a78e-4ce4-82d9-a8b237a9739d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:05.942477 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:06.007093 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8da86d-a78e-4ce4-82d9-a8b237a9739d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:06.040659 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85818e3e-a63d-472d-b564-98ab09d46542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:06.041511 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85818e3e-a63d-472d-b564-98ab09d46542 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:06.041796 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85818e3e-a63d-472d-b564-98ab09d46542 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:06.042070 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85818e3e-a63d-472d-b564-98ab09d46542 None admin] PATH_INFO: `/v3/users/a86ea826c5404521a1dc6b2162730f43` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:06.043275 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85818e3e-a63d-472d-b564-98ab09d46542 None admin] RBAC: Authorizing `identity:get_user(user_id=a86ea826c5404521a1dc6b2162730f43)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:06.044702 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85818e3e-a63d-472d-b564-98ab09d46542 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:06.046311 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 561/1121] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:22:05 2022] GET /identity/v3/users/a86ea826c5404521a1dc6b2162730f43 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:06.052216 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d4d97-3121-4175-bc8c-4178487ea5bd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:06.067486 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:06.150404 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d4d97-3121-4175-bc8c-4178487ea5bd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:06.183556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5145e6-097b-44c5-87f3-cef8746579ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:06.184342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5145e6-097b-44c5-87f3-cef8746579ae None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:06.184627 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5145e6-097b-44c5-87f3-cef8746579ae None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:06.184907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5145e6-097b-44c5-87f3-cef8746579ae None admin] PATH_INFO: `/v3/projects/9f8f17b758264cf38b4c2b5fe48194da` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:06.186070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5145e6-097b-44c5-87f3-cef8746579ae None admin] RBAC: Authorizing `identity:get_project(project_id=9f8f17b758264cf38b4c2b5fe48194da)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:06.187530 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5145e6-097b-44c5-87f3-cef8746579ae None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:06.189198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 561/1122] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:22:06 2022] GET /identity/v3/projects/9f8f17b758264cf38b4c2b5fe48194da => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:06.195234 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85818e3e-a63d-472d-b564-98ab09d46542 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:06.292915 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85818e3e-a63d-472d-b564-98ab09d46542 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:06.294410 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:06.326627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56d091-7bd4-4661-a239-6ca0cf9a5592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:06.327502 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56d091-7bd4-4661-a239-6ca0cf9a5592 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:06.327788 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56d091-7bd4-4661-a239-6ca0cf9a5592 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:06.328064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56d091-7bd4-4661-a239-6ca0cf9a5592 None admin] PATH_INFO: `/v3/projects/9f8f17b758264cf38b4c2b5fe48194da/users/a86ea826c5404521a1dc6b2162730f43/roles/6005bd32383a4706a69108f7a2cf3ed1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:06.330241 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56d091-7bd4-4661-a239-6ca0cf9a5592 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f8f17b758264cf38b4c2b5fe48194da, user_id=a86ea826c5404521a1dc6b2162730f43, role_id=6005bd32383a4706a69108f7a2cf3ed1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:06.331702 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56d091-7bd4-4661-a239-6ca0cf9a5592 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:06.344518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 562/1123] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 12:22:06 2022] PUT /identity/v3/projects/9f8f17b758264cf38b4c2b5fe48194da/users/a86ea826c5404521a1dc6b2162730f43/roles/6005bd32383a4706a69108f7a2cf3ed1 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:22:08.326778 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c883319-ac63-4507-bf0c-e972d54ba36e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:08.327310 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c883319-ac63-4507-bf0c-e972d54ba36e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:08.327404 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c883319-ac63-4507-bf0c-e972d54ba36e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:08.328702 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 562/1124] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:22:08.333763 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f9c43-6839-4f90-874a-c5112f86cbd8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:08.334147 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f9c43-6839-4f90-874a-c5112f86cbd8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:08.334314 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f9c43-6839-4f90-874a-c5112f86cbd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:08.350852 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7b0f9c43-6839-4f90-874a-c5112f86cbd8 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:22:08.352706 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7b0f9c43-6839-4f90-874a-c5112f86cbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:22:08.417109 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 563/1125] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:22:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:22:08.440715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c883319-ac63-4507-bf0c-e972d54ba36e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:08.519170 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:08.539872 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c883319-ac63-4507-bf0c-e972d54ba36e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:08.573047 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63977110-b29f-4451-836d-d8eaad40543a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:08.573885 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63977110-b29f-4451-836d-d8eaad40543a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:08.574169 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63977110-b29f-4451-836d-d8eaad40543a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:08.574473 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63977110-b29f-4451-836d-d8eaad40543a None admin] PATH_INFO: `/v3/roles/6005bd32383a4706a69108f7a2cf3ed1` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:08.575928 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63977110-b29f-4451-836d-d8eaad40543a None admin] RBAC: Authorizing `identity:get_role(role_id=6005bd32383a4706a69108f7a2cf3ed1)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:08.577335 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63977110-b29f-4451-836d-d8eaad40543a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:08.578669 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 563/1126] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:22:08 2022] GET /identity/v3/roles/6005bd32383a4706a69108f7a2cf3ed1 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:08.584280 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f9c43-6839-4f90-874a-c5112f86cbd8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:08.672219 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:08.682229 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f9c43-6839-4f90-874a-c5112f86cbd8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:08.715549 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d93eb3-8bd0-44ac-82c9-f894f20839fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:08.716411 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d93eb3-8bd0-44ac-82c9-f894f20839fa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:08.716694 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d93eb3-8bd0-44ac-82c9-f894f20839fa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:08.716972 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d93eb3-8bd0-44ac-82c9-f894f20839fa None admin] PATH_INFO: `/v3/users/a86ea826c5404521a1dc6b2162730f43` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:08.718145 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d93eb3-8bd0-44ac-82c9-f894f20839fa None admin] RBAC: Authorizing `identity:get_user(user_id=a86ea826c5404521a1dc6b2162730f43)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:08.719595 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d93eb3-8bd0-44ac-82c9-f894f20839fa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:08.721205 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 564/1127] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:22:08 2022] GET /identity/v3/users/a86ea826c5404521a1dc6b2162730f43 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:08.728093 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63977110-b29f-4451-836d-d8eaad40543a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:08.825511 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63977110-b29f-4451-836d-d8eaad40543a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:08.859994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9a326-9a44-4ed6-8d5d-0fad3dd61500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:08.860574 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9a326-9a44-4ed6-8d5d-0fad3dd61500 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:08.860858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9a326-9a44-4ed6-8d5d-0fad3dd61500 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:08.861134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf9a326-9a44-4ed6-8d5d-0fad3dd61500 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:08.878347 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:08.926864 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9a326-9a44-4ed6-8d5d-0fad3dd61500 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:08.928296 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf9a326-9a44-4ed6-8d5d-0fad3dd61500 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:09.020458 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 564/1128] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:22:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 12:22:09.026716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d93eb3-8bd0-44ac-82c9-f894f20839fa None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:09.123232 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d93eb3-8bd0-44ac-82c9-f894f20839fa None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:09.155570 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f194205-2be3-4cbf-993c-b97ada49629e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:09.156422 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f194205-2be3-4cbf-993c-b97ada49629e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:09.156709 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f194205-2be3-4cbf-993c-b97ada49629e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:09.156984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f194205-2be3-4cbf-993c-b97ada49629e None admin] PATH_INFO: `/v3/projects/9f8f17b758264cf38b4c2b5fe48194da` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:09.158181 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f194205-2be3-4cbf-993c-b97ada49629e None admin] RBAC: Authorizing `identity:get_project(project_id=9f8f17b758264cf38b4c2b5fe48194da)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:09.159642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f194205-2be3-4cbf-993c-b97ada49629e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:09.161354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 565/1129] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:22:09 2022] GET /identity/v3/projects/9f8f17b758264cf38b4c2b5fe48194da => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:09.168014 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9a326-9a44-4ed6-8d5d-0fad3dd61500 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:09.218916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:09.268940 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf9a326-9a44-4ed6-8d5d-0fad3dd61500 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:09.302734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d701741-1942-4a3d-a322-a566d6f4f2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:09.303372 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d701741-1942-4a3d-a322-a566d6f4f2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:09.303654 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d701741-1942-4a3d-a322-a566d6f4f2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:09.303929 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d701741-1942-4a3d-a322-a566d6f4f2b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:09.304780 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d701741-1942-4a3d-a322-a566d6f4f2b6 None admin] RBAC: Adding query filter params (role.id=6005bd32383a4706a69108f7a2cf3ed1, scope.project.id=9f8f17b758264cf38b4c2b5fe48194da, user.id=a86ea826c5404521a1dc6b2162730f43) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:22:09.305177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d701741-1942-4a3d-a322-a566d6f4f2b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:09.306737 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d701741-1942-4a3d-a322-a566d6f4f2b6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:09.312498 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 565/1130] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:22:09 2022] GET /identity/v3/role_assignments?user.id=a86ea826c5404521a1dc6b2162730f43&scope.project.id=9f8f17b758264cf38b4c2b5fe48194da&role.id=6005bd32383a4706a69108f7a2cf3ed1 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:11.345271 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5ce29-fb60-4a14-9144-3ac16dc6af88 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:11.345831 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5ce29-fb60-4a14-9144-3ac16dc6af88 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:11.345831 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5ce29-fb60-4a14-9144-3ac16dc6af88 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:11.347211 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 566/1131] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:22:11.352661 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ec110-9d84-4854-b1db-c664e57696d2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:11.353094 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ec110-9d84-4854-b1db-c664e57696d2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:11.353227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ec110-9d84-4854-b1db-c664e57696d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:11.369752 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a21ec110-9d84-4854-b1db-c664e57696d2 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:22:11.370994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a21ec110-9d84-4854-b1db-c664e57696d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:22:11.435026 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 566/1132] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:22:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 12:22:11.458575 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5ce29-fb60-4a14-9144-3ac16dc6af88 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:11.486636 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:11.558468 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5ce29-fb60-4a14-9144-3ac16dc6af88 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:11.591485 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929715e5-a0de-43db-a040-a356c1bcf9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:11.592271 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929715e5-a0de-43db-a040-a356c1bcf9ee None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:11.592562 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929715e5-a0de-43db-a040-a356c1bcf9ee None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:11.592841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929715e5-a0de-43db-a040-a356c1bcf9ee None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:11.594009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929715e5-a0de-43db-a040-a356c1bcf9ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:11.595528 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929715e5-a0de-43db-a040-a356c1bcf9ee None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:11.597082 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 567/1133] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:22:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:11.602978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ec110-9d84-4854-b1db-c664e57696d2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:11.644235 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:11.699440 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ec110-9d84-4854-b1db-c664e57696d2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:11.731501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a7b05-ba53-4c60-9e24-a0dd0be32ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:11.732147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a7b05-ba53-4c60-9e24-a0dd0be32ed6 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:11.732430 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a7b05-ba53-4c60-9e24-a0dd0be32ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:11.732707 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a7b05-ba53-4c60-9e24-a0dd0be32ed6 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:11.733414 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a7b05-ba53-4c60-9e24-a0dd0be32ed6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:11.734874 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a7b05-ba53-4c60-9e24-a0dd0be32ed6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:11.759799 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 567/1134] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jun 29 12:22:11 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:22:13.817339 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d7bec-0ffc-42c7-8f78-502c4576bc22 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:13.817858 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d7bec-0ffc-42c7-8f78-502c4576bc22 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:13.817970 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d7bec-0ffc-42c7-8f78-502c4576bc22 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:13.819292 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 568/1135] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:22:13.824741 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e3ebe-8023-4dfc-a548-ea308e1dbab9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:13.825119 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e3ebe-8023-4dfc-a548-ea308e1dbab9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:13.825251 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7e3ebe-8023-4dfc-a548-ea308e1dbab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:13.842845 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8a7e3ebe-8023-4dfc-a548-ea308e1dbab9 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:22:13.844011 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8a7e3ebe-8023-4dfc-a548-ea308e1dbab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:22:13.907782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 568/1136] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:22:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:22:13.933083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d7bec-0ffc-42c7-8f78-502c4576bc22 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:14.024328 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:14.031350 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d7bec-0ffc-42c7-8f78-502c4576bc22 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:14.064290 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87694399-7088-4481-9ec8-b021ee4d6e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:14.065059 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87694399-7088-4481-9ec8-b021ee4d6e58 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:14.065341 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87694399-7088-4481-9ec8-b021ee4d6e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:14.065612 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87694399-7088-4481-9ec8-b021ee4d6e58 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:14.066783 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87694399-7088-4481-9ec8-b021ee4d6e58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:14.068462 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87694399-7088-4481-9ec8-b021ee4d6e58 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:14.070601 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 569/1137] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:22:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:14.077054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e3ebe-8023-4dfc-a548-ea308e1dbab9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:14.101202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:14.175514 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7e3ebe-8023-4dfc-a548-ea308e1dbab9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:14.208037 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e1075-213c-4690-a350-7a5640f897ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:14.208689 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e1075-213c-4690-a350-7a5640f897ca None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:14.208978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e1075-213c-4690-a350-7a5640f897ca None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:14.209259 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e1075-213c-4690-a350-7a5640f897ca None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:14.209964 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e1075-213c-4690-a350-7a5640f897ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:14.211985 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e1075-213c-4690-a350-7a5640f897ca None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:14.231218 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 569/1138] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 12:22:14 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:22:16.265320 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba8668-8006-4886-ad40-cc8ace6a90bc None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:16.265869 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba8668-8006-4886-ad40-cc8ace6a90bc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:16.265869 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba8668-8006-4886-ad40-cc8ace6a90bc None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:16.267236 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 570/1139] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:22:16.272839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3224c73-ffd2-4d2a-ab80-42b25409587b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:16.273273 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3224c73-ffd2-4d2a-ab80-42b25409587b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:16.273405 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3224c73-ffd2-4d2a-ab80-42b25409587b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:16.289774 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a3224c73-ffd2-4d2a-ab80-42b25409587b None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:22:16.290979 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a3224c73-ffd2-4d2a-ab80-42b25409587b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:22:16.355102 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 570/1140] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:22:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:22:16.378624 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba8668-8006-4886-ad40-cc8ace6a90bc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:16.478123 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba8668-8006-4886-ad40-cc8ace6a90bc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:16.511091 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa2c04-28f1-4702-9360-9a3aaaa1a6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:16.511870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa2c04-28f1-4702-9360-9a3aaaa1a6bb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:16.512159 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa2c04-28f1-4702-9360-9a3aaaa1a6bb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:16.512443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa2c04-28f1-4702-9360-9a3aaaa1a6bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:16.522617 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aa2c04-28f1-4702-9360-9a3aaaa1a6bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:22:16.530296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-24aa2c04-28f1-4702-9360-9a3aaaa1a6bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:22:16.530296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:22:16.530296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:22:16.530296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:22:16.530296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:22:16.530296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:22:16.530296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:22:16.530296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:22:16.530296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:22:16.530296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:22:16.530296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:22:16.530296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:22:16.530296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:22:16.530296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:22:16.530296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:22:16.530296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:22:16.531357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:22:16.531357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:22:16.531357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:22:16.531357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:22:16.531357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:22:16.531357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:22:16.531357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:22:16.531357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:22:16.531357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:22:16.531357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:22:16.531357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:22:16.531357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:22:16.531357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:22:16.531357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:22:16.531357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:22:16.531357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:22:16.532316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:22:16.532316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:22:16.532316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:22:16.532316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:22:16.532316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:22:16.532316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:22:16.532316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:22:16.532316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:22:16.532316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:22:16.532316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:22:16.532316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:22:16.532316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:22:16.532316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 571/1141] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:22:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:22:16.537568 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3224c73-ffd2-4d2a-ab80-42b25409587b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:16.549476 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:16.634684 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3224c73-ffd2-4d2a-ab80-42b25409587b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:16.667260 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762993ec-7c68-4f56-808c-4d9136670cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:16.667910 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762993ec-7c68-4f56-808c-4d9136670cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:16.668192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762993ec-7c68-4f56-808c-4d9136670cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:16.668468 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762993ec-7c68-4f56-808c-4d9136670cd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:16.669201 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762993ec-7c68-4f56-808c-4d9136670cd3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:22:16.669508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762993ec-7c68-4f56-808c-4d9136670cd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:16.670900 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762993ec-7c68-4f56-808c-4d9136670cd3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:16.678695 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 571/1142] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:22:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:16.684580 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa2c04-28f1-4702-9360-9a3aaaa1a6bb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:16.709630 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:16.781716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aa2c04-28f1-4702-9360-9a3aaaa1a6bb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:16.813808 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acc4fa-0ac9-4c3b-86d2-eb1845662b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:16.814683 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acc4fa-0ac9-4c3b-86d2-eb1845662b8f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:16.814968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acc4fa-0ac9-4c3b-86d2-eb1845662b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:16.815242 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acc4fa-0ac9-4c3b-86d2-eb1845662b8f None admin] PATH_INFO: `/v3/users/eda47af00db84d10b0ed58aab6334bd2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:16.827906 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acc4fa-0ac9-4c3b-86d2-eb1845662b8f None admin] RBAC: Authorizing `identity:get_user(user_id=eda47af00db84d10b0ed58aab6334bd2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:16.829328 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83acc4fa-0ac9-4c3b-86d2-eb1845662b8f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:16.830949 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 572/1143] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:22:16 2022] GET /identity/v3/users/eda47af00db84d10b0ed58aab6334bd2 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:16.837566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762993ec-7c68-4f56-808c-4d9136670cd3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:16.905453 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:16.934293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762993ec-7c68-4f56-808c-4d9136670cd3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:16.966488 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414d305-f31f-4898-afef-3b503ddd8c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:16.967127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414d305-f31f-4898-afef-3b503ddd8c42 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:16.967411 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414d305-f31f-4898-afef-3b503ddd8c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:16.967689 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414d305-f31f-4898-afef-3b503ddd8c42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:17.032614 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414d305-f31f-4898-afef-3b503ddd8c42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:17.033997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414d305-f31f-4898-afef-3b503ddd8c42 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:17.098481 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:17.128770 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 572/1144] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:22:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 12:22:17.135491 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acc4fa-0ac9-4c3b-86d2-eb1845662b8f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:17.214998 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:17.232914 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acc4fa-0ac9-4c3b-86d2-eb1845662b8f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:17.266256 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a201b-1526-479e-8390-938a7fbbedf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:17.267137 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a201b-1526-479e-8390-938a7fbbedf9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:17.267423 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a201b-1526-479e-8390-938a7fbbedf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:17.267702 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a201b-1526-479e-8390-938a7fbbedf9 None admin] PATH_INFO: `/v3/projects/54d8cf88d0ae4fce9f2d18be87a5a9cd` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:17.268871 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0a201b-1526-479e-8390-938a7fbbedf9 None admin] RBAC: Authorizing `identity:get_project(project_id=54d8cf88d0ae4fce9f2d18be87a5a9cd)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:17.270308 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0a201b-1526-479e-8390-938a7fbbedf9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:17.272296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 573/1145] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:22:17 2022] GET /identity/v3/projects/54d8cf88d0ae4fce9f2d18be87a5a9cd => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:17.278999 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414d305-f31f-4898-afef-3b503ddd8c42 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:17.377089 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414d305-f31f-4898-afef-3b503ddd8c42 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:17.413117 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00739e-145e-4e6a-b425-c38603d9b30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:17.413764 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00739e-145e-4e6a-b425-c38603d9b30a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:17.414106 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00739e-145e-4e6a-b425-c38603d9b30a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:17.414359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00739e-145e-4e6a-b425-c38603d9b30a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:17.415379 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00739e-145e-4e6a-b425-c38603d9b30a None admin] RBAC: Adding query filter params (role.id=3a7f4bf556344407a6d9540c89d13641, scope.project.id=54d8cf88d0ae4fce9f2d18be87a5a9cd, user.id=eda47af00db84d10b0ed58aab6334bd2) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:22:17.415691 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00739e-145e-4e6a-b425-c38603d9b30a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:17.417110 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00739e-145e-4e6a-b425-c38603d9b30a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:17.422345 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 573/1146] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:22:17 2022] GET /identity/v3/role_assignments?user.id=eda47af00db84d10b0ed58aab6334bd2&scope.project.id=54d8cf88d0ae4fce9f2d18be87a5a9cd&role.id=3a7f4bf556344407a6d9540c89d13641 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:19.422652 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ce0d6-a93d-42e6-9012-fb4f6d7c8e9e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:19.423174 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ce0d6-a93d-42e6-9012-fb4f6d7c8e9e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:19.423289 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ce0d6-a93d-42e6-9012-fb4f6d7c8e9e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:19.424602 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 574/1147] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:22:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:22:19.430145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577c138-26d2-4c96-bb73-36350d5834b1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:19.430619 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577c138-26d2-4c96-bb73-36350d5834b1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:19.430730 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1577c138-26d2-4c96-bb73-36350d5834b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:19.447616 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1577c138-26d2-4c96-bb73-36350d5834b1 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:22:19.449158 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1577c138-26d2-4c96-bb73-36350d5834b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:22:19.477142 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:19.513807 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 574/1148] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:22:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:22:19.537305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ce0d6-a93d-42e6-9012-fb4f6d7c8e9e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:19.649276 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ce0d6-a93d-42e6-9012-fb4f6d7c8e9e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:19.682048 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cef752-6399-4fbf-9a44-e914b926c146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:19.682869 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cef752-6399-4fbf-9a44-e914b926c146 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:19.683160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cef752-6399-4fbf-9a44-e914b926c146 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:19.683442 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cef752-6399-4fbf-9a44-e914b926c146 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:19.693306 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cef752-6399-4fbf-9a44-e914b926c146 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:22:19.701536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a0cef752-6399-4fbf-9a44-e914b926c146 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:22:19.701536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:22:19.701536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:22:19.701536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:22:19.701536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:22:19.701536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:22:19.701536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:22:19.701536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:22:19.701536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:22:19.701536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:22:19.701536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:22:19.701536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:22:19.701536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:22:19.701536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:22:19.701536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:22:19.701536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:22:19.702655 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:22:19.702655 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:22:19.702655 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:22:19.702655 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:22:19.702655 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:22:19.702655 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:22:19.702655 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:22:19.702655 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:22:19.702655 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:22:19.702655 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:22:19.702655 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:22:19.702655 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:22:19.702655 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:22:19.702655 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:22:19.702655 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:22:19.702655 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:22:19.703779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:22:19.703779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:22:19.703779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:22:19.703779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:22:19.703779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:22:19.703779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:22:19.703779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:22:19.703779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:22:19.703779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:22:19.703779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:22:19.703779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:22:19.703779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:22:19.703779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 575/1149] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:22:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:22:19.709216 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577c138-26d2-4c96-bb73-36350d5834b1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:19.805125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1577c138-26d2-4c96-bb73-36350d5834b1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:19.837714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745c560-e01c-4f70-9000-d88697934c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:19.838369 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745c560-e01c-4f70-9000-d88697934c70 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:19.838678 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745c560-e01c-4f70-9000-d88697934c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:19.838956 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745c560-e01c-4f70-9000-d88697934c70 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:19.839692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745c560-e01c-4f70-9000-d88697934c70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:22:19.839997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745c560-e01c-4f70-9000-d88697934c70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:19.841354 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745c560-e01c-4f70-9000-d88697934c70 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:19.849170 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 575/1150] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:22:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:19.855347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cef752-6399-4fbf-9a44-e914b926c146 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:19.869415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:19.957902 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cef752-6399-4fbf-9a44-e914b926c146 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:19.989902 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef14fe-8acd-45f2-b116-95f1ea7cb9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:19.990795 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef14fe-8acd-45f2-b116-95f1ea7cb9de None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:19.991079 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef14fe-8acd-45f2-b116-95f1ea7cb9de None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:19.991354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef14fe-8acd-45f2-b116-95f1ea7cb9de None admin] PATH_INFO: `/v3/users/eda47af00db84d10b0ed58aab6334bd2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:19.993193 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef14fe-8acd-45f2-b116-95f1ea7cb9de None admin] RBAC: Authorizing `identity:get_user(user_id=eda47af00db84d10b0ed58aab6334bd2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:19.994635 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef14fe-8acd-45f2-b116-95f1ea7cb9de None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:19.996246 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 576/1151] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:22:19 2022] GET /identity/v3/users/eda47af00db84d10b0ed58aab6334bd2 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:20.002183 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745c560-e01c-4f70-9000-d88697934c70 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:20.030692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:20.098721 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745c560-e01c-4f70-9000-d88697934c70 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:20.130652 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e140f64a-6986-4c94-be32-dfc44c2ae78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:20.131506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140f64a-6986-4c94-be32-dfc44c2ae78f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:20.131791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140f64a-6986-4c94-be32-dfc44c2ae78f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:20.132069 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140f64a-6986-4c94-be32-dfc44c2ae78f None admin] PATH_INFO: `/v3/projects/54d8cf88d0ae4fce9f2d18be87a5a9cd` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:20.134123 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e140f64a-6986-4c94-be32-dfc44c2ae78f None admin] RBAC: Authorizing `identity:get_project(project_id=54d8cf88d0ae4fce9f2d18be87a5a9cd)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:20.135591 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e140f64a-6986-4c94-be32-dfc44c2ae78f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:20.137303 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 576/1152] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:22:20 2022] GET /identity/v3/projects/54d8cf88d0ae4fce9f2d18be87a5a9cd => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:20.143086 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef14fe-8acd-45f2-b116-95f1ea7cb9de None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:20.222476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:20.244229 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef14fe-8acd-45f2-b116-95f1ea7cb9de None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:20.277923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6391c49-6899-4145-9512-384743af6b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:20.278845 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6391c49-6899-4145-9512-384743af6b3f None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:20.279133 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6391c49-6899-4145-9512-384743af6b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:20.279414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6391c49-6899-4145-9512-384743af6b3f None admin] PATH_INFO: `/v3/projects/54d8cf88d0ae4fce9f2d18be87a5a9cd/users/eda47af00db84d10b0ed58aab6334bd2/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:20.281335 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6391c49-6899-4145-9512-384743af6b3f None admin] RBAC: Authorizing `identity:create_grant(project_id=54d8cf88d0ae4fce9f2d18be87a5a9cd, user_id=eda47af00db84d10b0ed58aab6334bd2, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:20.282781 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6391c49-6899-4145-9512-384743af6b3f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:20.295925 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 577/1153] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 12:22:20 2022] PUT /identity/v3/projects/54d8cf88d0ae4fce9f2d18be87a5a9cd/users/eda47af00db84d10b0ed58aab6334bd2/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:22:22.270117 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1741e0-172d-4bad-a570-f434d43f4e8f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:22.270695 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1741e0-172d-4bad-a570-f434d43f4e8f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:22.270800 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1741e0-172d-4bad-a570-f434d43f4e8f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:22.272419 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 577/1154] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:22:22.278051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00f655-b1eb-49d3-a69a-501af897cc5a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:22.278478 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00f655-b1eb-49d3-a69a-501af897cc5a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:22.278679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae00f655-b1eb-49d3-a69a-501af897cc5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:22.295889 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ae00f655-b1eb-49d3-a69a-501af897cc5a None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:22:22.297064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ae00f655-b1eb-49d3-a69a-501af897cc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:22:22.362777 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 578/1155] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:22:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:22:22.387624 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1741e0-172d-4bad-a570-f434d43f4e8f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:22.479930 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:22.486514 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1741e0-172d-4bad-a570-f434d43f4e8f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:22.518790 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be9656-a511-48e4-b505-91f45566ed6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:22.519560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be9656-a511-48e4-b505-91f45566ed6a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:22.519846 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be9656-a511-48e4-b505-91f45566ed6a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:22.520175 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00be9656-a511-48e4-b505-91f45566ed6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:22.531781 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00be9656-a511-48e4-b505-91f45566ed6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:22:22.539512 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-00be9656-a511-48e4-b505-91f45566ed6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:22:22.539512 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:22:22.539512 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:22:22.539512 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:22:22.539512 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:22:22.539512 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:22:22.539512 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:22:22.539512 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:22:22.539512 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:22:22.539512 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:22:22.539512 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:22:22.539512 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:22:22.539512 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:22:22.539512 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:22:22.539512 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:22:22.539512 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:22:22.540647 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:22:22.540647 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:22:22.540647 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:22:22.540647 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:22:22.540647 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:22:22.540647 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:22:22.540647 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:22:22.540647 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:22:22.540647 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:22:22.540647 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:22:22.540647 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:22:22.540647 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:22:22.540647 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:22:22.540647 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:22:22.540647 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:22:22.540647 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:22:22.546566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:22:22.546566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:22:22.546566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:22:22.546566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:22:22.546566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:22:22.546566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:22:22.546566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:22:22.546566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:22:22.546566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:22:22.546566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:22:22.546566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:22:22.546566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:22:22.546566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 578/1156] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:22:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:22:22.547448 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00f655-b1eb-49d3-a69a-501af897cc5a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:22.610987 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:22.644507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae00f655-b1eb-49d3-a69a-501af897cc5a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:22.678126 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b539e1-1682-4769-8a17-f7e13763cda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:22.678813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b539e1-1682-4769-8a17-f7e13763cda7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:22.679103 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b539e1-1682-4769-8a17-f7e13763cda7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:22.679405 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b539e1-1682-4769-8a17-f7e13763cda7 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:22.680151 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b539e1-1682-4769-8a17-f7e13763cda7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:22:22.680461 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b539e1-1682-4769-8a17-f7e13763cda7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:22.681821 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b539e1-1682-4769-8a17-f7e13763cda7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:22.689610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 579/1157] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:22:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:22:22.696137 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be9656-a511-48e4-b505-91f45566ed6a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:22.794479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00be9656-a511-48e4-b505-91f45566ed6a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:22.827551 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb777b7-0318-4a2b-8171-98b1cb6e97cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:22.828405 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb777b7-0318-4a2b-8171-98b1cb6e97cd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:22.828688 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb777b7-0318-4a2b-8171-98b1cb6e97cd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:22.828966 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb777b7-0318-4a2b-8171-98b1cb6e97cd None admin] PATH_INFO: `/v3/users/eda47af00db84d10b0ed58aab6334bd2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:22.830130 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb777b7-0318-4a2b-8171-98b1cb6e97cd None admin] RBAC: Authorizing `identity:get_user(user_id=eda47af00db84d10b0ed58aab6334bd2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:22.831593 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb777b7-0318-4a2b-8171-98b1cb6e97cd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:22.833204 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 579/1158] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:22:22 2022] GET /identity/v3/users/eda47af00db84d10b0ed58aab6334bd2 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:22.839426 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b539e1-1682-4769-8a17-f7e13763cda7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:22.936524 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b539e1-1682-4769-8a17-f7e13763cda7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:22.963841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:22.969965 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db4e14-4ec7-4ef2-a2fe-7e7007d9634b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:22.970647 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db4e14-4ec7-4ef2-a2fe-7e7007d9634b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:22.970939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db4e14-4ec7-4ef2-a2fe-7e7007d9634b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:22.971222 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db4e14-4ec7-4ef2-a2fe-7e7007d9634b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:23.036442 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0db4e14-4ec7-4ef2-a2fe-7e7007d9634b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:23.037881 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0db4e14-4ec7-4ef2-a2fe-7e7007d9634b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:23.134302 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 580/1159] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:22:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 12:22:23.140114 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb777b7-0318-4a2b-8171-98b1cb6e97cd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:23.165029 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:23.235231 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb777b7-0318-4a2b-8171-98b1cb6e97cd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:23.267172 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8e898-615c-48fe-a3c4-f5616890a3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:23.268005 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8e898-615c-48fe-a3c4-f5616890a3e1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:23.268284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8e898-615c-48fe-a3c4-f5616890a3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:23.268556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8e898-615c-48fe-a3c4-f5616890a3e1 None admin] PATH_INFO: `/v3/projects/54d8cf88d0ae4fce9f2d18be87a5a9cd` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:23.269703 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8e898-615c-48fe-a3c4-f5616890a3e1 None admin] RBAC: Authorizing `identity:get_project(project_id=54d8cf88d0ae4fce9f2d18be87a5a9cd)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:23.271149 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8e898-615c-48fe-a3c4-f5616890a3e1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:23.273162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 580/1160] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:22:23 2022] GET /identity/v3/projects/54d8cf88d0ae4fce9f2d18be87a5a9cd => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:23.279086 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db4e14-4ec7-4ef2-a2fe-7e7007d9634b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:23.308429 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:23.375265 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0db4e14-4ec7-4ef2-a2fe-7e7007d9634b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:23.408078 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f04593-5e70-404a-a632-7bb75d199f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:23.408663 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f04593-5e70-404a-a632-7bb75d199f3c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:23.408945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f04593-5e70-404a-a632-7bb75d199f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:23.409218 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f04593-5e70-404a-a632-7bb75d199f3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:23.410049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f04593-5e70-404a-a632-7bb75d199f3c None admin] RBAC: Adding query filter params (role.id=3a7f4bf556344407a6d9540c89d13641, scope.project.id=54d8cf88d0ae4fce9f2d18be87a5a9cd, user.id=eda47af00db84d10b0ed58aab6334bd2) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:22:23.410360 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f04593-5e70-404a-a632-7bb75d199f3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:23.411792 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f04593-5e70-404a-a632-7bb75d199f3c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:23.419230 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 581/1161] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:22:23 2022] GET /identity/v3/role_assignments?user.id=eda47af00db84d10b0ed58aab6334bd2&scope.project.id=54d8cf88d0ae4fce9f2d18be87a5a9cd&role.id=3a7f4bf556344407a6d9540c89d13641 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:25.441474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c39054-2bba-4d41-b840-9cc471a92f79 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:25.441990 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c39054-2bba-4d41-b840-9cc471a92f79 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:25.442103 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c39054-2bba-4d41-b840-9cc471a92f79 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:25.443426 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 581/1162] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:22:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:22:25.448569 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe0701-2d75-491d-a19a-ed04557de083 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:25.448975 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe0701-2d75-491d-a19a-ed04557de083 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:25.449120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe0701-2d75-491d-a19a-ed04557de083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:25.465442 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4abe0701-2d75-491d-a19a-ed04557de083 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:22:25.466626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4abe0701-2d75-491d-a19a-ed04557de083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:22:25.530438 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 582/1163] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:22:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:22:25.552853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c39054-2bba-4d41-b840-9cc471a92f79 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:25.642454 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:25.652983 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c39054-2bba-4d41-b840-9cc471a92f79 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:25.686020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b197263-c1fc-44a3-8b6e-e703338880e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:25.686683 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b197263-c1fc-44a3-8b6e-e703338880e6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:25.686974 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b197263-c1fc-44a3-8b6e-e703338880e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:25.687255 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b197263-c1fc-44a3-8b6e-e703338880e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:25.751501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b197263-c1fc-44a3-8b6e-e703338880e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:25.752943 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b197263-c1fc-44a3-8b6e-e703338880e6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:25.839084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:25.848461 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 582/1164] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:22:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 12:22:25.854959 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe0701-2d75-491d-a19a-ed04557de083 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:25.884812 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:25.958354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe0701-2d75-491d-a19a-ed04557de083 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:25.990370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b2d61a-6744-450c-a8a2-206d1dae98b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:25.991187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2d61a-6744-450c-a8a2-206d1dae98b4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:25.991471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2d61a-6744-450c-a8a2-206d1dae98b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:25.991746 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b2d61a-6744-450c-a8a2-206d1dae98b4 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:25.998809 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b2d61a-6744-450c-a8a2-206d1dae98b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:26.000244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b2d61a-6744-450c-a8a2-206d1dae98b4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:26.014503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-06b2d61a-6744-450c-a8a2-206d1dae98b4 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 29 12:22:26.014503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:22:26.014503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 29 12:22:26.014503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 29 12:22:26.014503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:22:26.014503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:22:26.014503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:22:26.014503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:22:26.014503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 29 12:22:26.014503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:22:26.014503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:22:26.014503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:22:26.014503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:22:26.014503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:22:26.014503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:22:26.014503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:22:26.014503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:22:26.015704 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:22:26.015704 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:22:26.015704 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:22:26.015704 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:22:26.015704 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:22:26.015704 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:22:26.015704 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:22:26.015704 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:22:26.015704 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:22:26.015704 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:22:26.015704 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:22:26.015704 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:22:26.015704 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:22:26.015704 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:22:26.015704 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:22:26.015704 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:22:26.016817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:22:26.016817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:22:26.016817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:22:26.016817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:22:26.016817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:22:26.016817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:22:26.016817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:22:26.016817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:22:26.016817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:22:26.016817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:22:26.016817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 29 12:22:26.016817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:22:26.016817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 29 12:22:26.016817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:22:26.016817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 583/1165] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jun 29 12:22:25 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:22:26.022003 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b197263-c1fc-44a3-8b6e-e703338880e6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:26.116534 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b197263-c1fc-44a3-8b6e-e703338880e6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:26.150506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500d75e-45e7-403c-905b-23ca6455ae43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:26.151150 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500d75e-45e7-403c-905b-23ca6455ae43 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:26.151430 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500d75e-45e7-403c-905b-23ca6455ae43 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:26.151704 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500d75e-45e7-403c-905b-23ca6455ae43 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:26.152412 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c500d75e-45e7-403c-905b-23ca6455ae43 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:22:26.152716 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c500d75e-45e7-403c-905b-23ca6455ae43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:26.154068 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c500d75e-45e7-403c-905b-23ca6455ae43 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:26.160930 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 583/1166] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Jun 29 12:22:26 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:28.170078 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b106041-8983-481f-a056-fc92cdd555ca None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:28.170629 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b106041-8983-481f-a056-fc92cdd555ca None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:28.170720 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b106041-8983-481f-a056-fc92cdd555ca None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:28.172040 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 584/1167] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:22:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:22:28.177463 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d3246-b58d-4f7b-ad8d-2590221bd4cd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:28.177870 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d3246-b58d-4f7b-ad8d-2590221bd4cd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:28.178025 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3d3246-b58d-4f7b-ad8d-2590221bd4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:28.194847 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-da3d3246-b58d-4f7b-ad8d-2590221bd4cd None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:22:28.196591 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-da3d3246-b58d-4f7b-ad8d-2590221bd4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:22:28.244891 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:28.261867 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 584/1168] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:22:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:22:28.284601 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b106041-8983-481f-a056-fc92cdd555ca None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:28.366415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:28.384002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b106041-8983-481f-a056-fc92cdd555ca None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:28.416239 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49e48a5-241e-47b8-acde-f1482c0cd2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:28.416890 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e48a5-241e-47b8-acde-f1482c0cd2fe None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:28.417174 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e48a5-241e-47b8-acde-f1482c0cd2fe None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:28.417450 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49e48a5-241e-47b8-acde-f1482c0cd2fe None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:28.418193 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49e48a5-241e-47b8-acde-f1482c0cd2fe None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:28.419589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49e48a5-241e-47b8-acde-f1482c0cd2fe None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:28.433519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 585/1169] 10.222.0.52 () {60 vars in 1138 bytes} [Wed Jun 29 12:22:28 2022] POST /identity/v3/domains => generated 267 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:22:30.448254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb09b3e-e4a0-4670-8e94-ba87bee4c969 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:30.448746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb09b3e-e4a0-4670-8e94-ba87bee4c969 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:30.448867 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb09b3e-e4a0-4670-8e94-ba87bee4c969 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:30.450197 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 585/1170] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:22:30.455431 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c24a81-c3f9-4ca6-8397-cf30a35fdc6b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:30.455830 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c24a81-c3f9-4ca6-8397-cf30a35fdc6b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:30.456004 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c24a81-c3f9-4ca6-8397-cf30a35fdc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:30.473413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b2c24a81-c3f9-4ca6-8397-cf30a35fdc6b None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:22:30.474631 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b2c24a81-c3f9-4ca6-8397-cf30a35fdc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:22:30.540990 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 586/1171] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:22:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:22:30.564567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb09b3e-e4a0-4670-8e94-ba87bee4c969 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:30.667493 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb09b3e-e4a0-4670-8e94-ba87bee4c969 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:30.700788 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4efd17a-e80b-46fa-9d1c-d207683ea344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:30.701640 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4efd17a-e80b-46fa-9d1c-d207683ea344 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:30.701926 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4efd17a-e80b-46fa-9d1c-d207683ea344 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:30.702205 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4efd17a-e80b-46fa-9d1c-d207683ea344 None admin] PATH_INFO: `/v3/domains/cd306200454a4d23a2560285e07df94f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:30.712258 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4efd17a-e80b-46fa-9d1c-d207683ea344 None admin] RBAC: Authorizing `identity:get_domain(domain_id=cd306200454a4d23a2560285e07df94f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:30.713700 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4efd17a-e80b-46fa-9d1c-d207683ea344 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:30.715368 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 586/1172] 10.222.0.52 () {56 vars in 1183 bytes} [Wed Jun 29 12:22:30 2022] GET /identity/v3/domains/cd306200454a4d23a2560285e07df94f => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:30.721218 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c24a81-c3f9-4ca6-8397-cf30a35fdc6b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:30.792521 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:30.819762 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c24a81-c3f9-4ca6-8397-cf30a35fdc6b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:30.853797 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f16018d-7963-4563-8d67-06192f126b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:30.854479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16018d-7963-4563-8d67-06192f126b28 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:30.854766 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16018d-7963-4563-8d67-06192f126b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:30.855043 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16018d-7963-4563-8d67-06192f126b28 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:30.855745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f16018d-7963-4563-8d67-06192f126b28 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:30.857172 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f16018d-7963-4563-8d67-06192f126b28 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:30.881945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 587/1173] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jun 29 12:22:30 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:22:32.932943 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f151b3-9024-43e7-8975-f77ed29e4615 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:32.933500 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f151b3-9024-43e7-8975-f77ed29e4615 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:32.933500 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f151b3-9024-43e7-8975-f77ed29e4615 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:32.934908 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 587/1174] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:22:32.940071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e4b72-a739-406f-b3fb-a72b23e78c50 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:32.940481 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e4b72-a739-406f-b3fb-a72b23e78c50 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:32.940640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e4b72-a739-406f-b3fb-a72b23e78c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:32.957416 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e06e4b72-a739-406f-b3fb-a72b23e78c50 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:22:32.958613 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e06e4b72-a739-406f-b3fb-a72b23e78c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:22:33.024865 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 588/1175] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:22:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:22:33.049949 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f151b3-9024-43e7-8975-f77ed29e4615 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:33.096988 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:33.150234 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f151b3-9024-43e7-8975-f77ed29e4615 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:33.181660 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea1b1a-c89a-43d6-b80d-b70c75dba825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:33.182526 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea1b1a-c89a-43d6-b80d-b70c75dba825 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:33.182812 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea1b1a-c89a-43d6-b80d-b70c75dba825 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:33.183136 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea1b1a-c89a-43d6-b80d-b70c75dba825 None admin] PATH_INFO: `/v3/domains/cd306200454a4d23a2560285e07df94f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:33.184256 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea1b1a-c89a-43d6-b80d-b70c75dba825 None admin] RBAC: Authorizing `identity:get_domain(domain_id=cd306200454a4d23a2560285e07df94f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:33.185682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea1b1a-c89a-43d6-b80d-b70c75dba825 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:33.187309 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 588/1176] 10.222.0.52 () {56 vars in 1183 bytes} [Wed Jun 29 12:22:33 2022] GET /identity/v3/domains/cd306200454a4d23a2560285e07df94f => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:33.193025 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e4b72-a739-406f-b3fb-a72b23e78c50 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:33.251074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:33.293778 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e4b72-a739-406f-b3fb-a72b23e78c50 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:33.326083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f2b49-e1d3-461d-883f-6f9bb911873e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:33.326768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f2b49-e1d3-461d-883f-6f9bb911873e None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:33.327051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f2b49-e1d3-461d-883f-6f9bb911873e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:33.327326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f2b49-e1d3-461d-883f-6f9bb911873e None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:33.328029 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7f2b49-e1d3-461d-883f-6f9bb911873e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:33.329436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7f2b49-e1d3-461d-883f-6f9bb911873e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:33.348791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 589/1177] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 12:22:33 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:22:35.461307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c1cc8-9c2a-4893-9e33-19cac5a80790 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:35.461846 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c1cc8-9c2a-4893-9e33-19cac5a80790 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:35.461846 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c1cc8-9c2a-4893-9e33-19cac5a80790 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:35.463267 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 589/1178] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:22:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:22:35.468430 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbcded-bb14-466e-b305-d59b6c235422 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:35.468813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbcded-bb14-466e-b305-d59b6c235422 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:35.468987 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbcded-bb14-466e-b305-d59b6c235422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:35.485681 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-22fbcded-bb14-466e-b305-d59b6c235422 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:22:35.486884 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-22fbcded-bb14-466e-b305-d59b6c235422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:22:35.551843 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 590/1179] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:22:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:22:35.575253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c1cc8-9c2a-4893-9e33-19cac5a80790 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:35.674280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c1cc8-9c2a-4893-9e33-19cac5a80790 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:35.687078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:35.711173 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72902bd-a4b5-49b6-ac4e-3bc3a11cddf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:35.711965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72902bd-a4b5-49b6-ac4e-3bc3a11cddf4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:35.712245 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72902bd-a4b5-49b6-ac4e-3bc3a11cddf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:35.712519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72902bd-a4b5-49b6-ac4e-3bc3a11cddf4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:35.722919 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72902bd-a4b5-49b6-ac4e-3bc3a11cddf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:22:35.731435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e72902bd-a4b5-49b6-ac4e-3bc3a11cddf4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:22:35.731435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:22:35.731435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:22:35.731435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:22:35.731435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:22:35.731435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:22:35.731435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:22:35.731435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:22:35.731435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:22:35.731435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:22:35.731435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:22:35.731435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:22:35.731435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:22:35.731435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:22:35.731435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:22:35.731435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:22:35.732733 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:22:35.732733 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:22:35.732733 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:22:35.732733 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:22:35.732733 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:22:35.732733 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:22:35.732733 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:22:35.732733 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:22:35.732733 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:22:35.732733 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:22:35.732733 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:22:35.732733 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:22:35.732733 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:22:35.732733 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:22:35.732733 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:22:35.732733 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:22:35.733849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:22:35.733849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:22:35.733849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:22:35.733849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:22:35.733849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:22:35.733849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:22:35.733849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:22:35.733849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:22:35.733849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:22:35.733849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:22:35.733849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:22:35.733849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:22:35.733849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 590/1180] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:22:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:22:35.738832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fbcded-bb14-466e-b305-d59b6c235422 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:35.780294 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:35.834657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fbcded-bb14-466e-b305-d59b6c235422 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:35.867960 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea032585-913f-4250-974d-a3e0aa4ed21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:35.868617 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea032585-913f-4250-974d-a3e0aa4ed21e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:35.868908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea032585-913f-4250-974d-a3e0aa4ed21e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:35.869192 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea032585-913f-4250-974d-a3e0aa4ed21e None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:35.869940 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea032585-913f-4250-974d-a3e0aa4ed21e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:22:35.870254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea032585-913f-4250-974d-a3e0aa4ed21e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:35.871660 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea032585-913f-4250-974d-a3e0aa4ed21e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:35.879532 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 591/1181] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:22:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:35.885722 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72902bd-a4b5-49b6-ac4e-3bc3a11cddf4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:35.981777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72902bd-a4b5-49b6-ac4e-3bc3a11cddf4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:36.013767 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2047b-387e-4dbd-b97b-97a473cf0031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:36.014641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2047b-387e-4dbd-b97b-97a473cf0031 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:36.014925 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2047b-387e-4dbd-b97b-97a473cf0031 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:36.015200 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2047b-387e-4dbd-b97b-97a473cf0031 None admin] PATH_INFO: `/v3/users/8d042d78ac9940f3b22756f17d59d95b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:36.028664 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2047b-387e-4dbd-b97b-97a473cf0031 None admin] RBAC: Authorizing `identity:get_user(user_id=8d042d78ac9940f3b22756f17d59d95b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:36.030096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2047b-387e-4dbd-b97b-97a473cf0031 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:36.031770 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 591/1182] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:22:35 2022] GET /identity/v3/users/8d042d78ac9940f3b22756f17d59d95b => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:36.038174 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea032585-913f-4250-974d-a3e0aa4ed21e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:36.137181 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea032585-913f-4250-974d-a3e0aa4ed21e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:36.139490 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:36.170606 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf43948-1c2a-419e-8453-9ede2a7254b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:36.171248 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf43948-1c2a-419e-8453-9ede2a7254b0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:36.171532 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf43948-1c2a-419e-8453-9ede2a7254b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:36.171810 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf43948-1c2a-419e-8453-9ede2a7254b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:36.237484 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf43948-1c2a-419e-8453-9ede2a7254b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:36.238893 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf43948-1c2a-419e-8453-9ede2a7254b0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:36.333034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 592/1183] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:22:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 12:22:36.345909 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2047b-387e-4dbd-b97b-97a473cf0031 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:36.376626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:36.443619 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2047b-387e-4dbd-b97b-97a473cf0031 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:36.478310 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12658d61-8cad-4def-a390-239c2ff7e927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:36.479190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12658d61-8cad-4def-a390-239c2ff7e927 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:36.479474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12658d61-8cad-4def-a390-239c2ff7e927 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:36.479753 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12658d61-8cad-4def-a390-239c2ff7e927 None admin] PATH_INFO: `/v3/projects/c2dc36cebcaa480e99868c8e9815d1bd` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:36.480915 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12658d61-8cad-4def-a390-239c2ff7e927 None admin] RBAC: Authorizing `identity:get_project(project_id=c2dc36cebcaa480e99868c8e9815d1bd)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:36.482337 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12658d61-8cad-4def-a390-239c2ff7e927 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:36.484042 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 592/1184] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:22:36 2022] GET /identity/v3/projects/c2dc36cebcaa480e99868c8e9815d1bd => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:36.490189 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf43948-1c2a-419e-8453-9ede2a7254b0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:36.493707 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:36.586652 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf43948-1c2a-419e-8453-9ede2a7254b0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:36.618887 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63452f5e-5a5d-4d8d-b2fa-154860eeddec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:36.619525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63452f5e-5a5d-4d8d-b2fa-154860eeddec None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:36.619809 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63452f5e-5a5d-4d8d-b2fa-154860eeddec None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:36.620087 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63452f5e-5a5d-4d8d-b2fa-154860eeddec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:36.620909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63452f5e-5a5d-4d8d-b2fa-154860eeddec None admin] RBAC: Adding query filter params (role.id=3a7f4bf556344407a6d9540c89d13641, scope.project.id=c2dc36cebcaa480e99868c8e9815d1bd, user.id=8d042d78ac9940f3b22756f17d59d95b) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:22:36.621220 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63452f5e-5a5d-4d8d-b2fa-154860eeddec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:36.622642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63452f5e-5a5d-4d8d-b2fa-154860eeddec None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:36.627869 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 593/1185] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:22:36 2022] GET /identity/v3/role_assignments?user.id=8d042d78ac9940f3b22756f17d59d95b&scope.project.id=c2dc36cebcaa480e99868c8e9815d1bd&role.id=3a7f4bf556344407a6d9540c89d13641 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:38.620128 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144e6ae-b7e4-4d45-88db-ef1e5995a8a2 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:38.620635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144e6ae-b7e4-4d45-88db-ef1e5995a8a2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:38.620635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144e6ae-b7e4-4d45-88db-ef1e5995a8a2 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:38.621982 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 593/1186] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:22:38.627169 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a86a0df-f979-4766-bef7-1f38c76128fc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:38.627580 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a86a0df-f979-4766-bef7-1f38c76128fc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:38.627727 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a86a0df-f979-4766-bef7-1f38c76128fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:38.645035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9a86a0df-f979-4766-bef7-1f38c76128fc None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:22:38.646211 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9a86a0df-f979-4766-bef7-1f38c76128fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:22:38.707330 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:38.712970 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 594/1187] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:22:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 12:22:38.735359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144e6ae-b7e4-4d45-88db-ef1e5995a8a2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:38.830483 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:38.834729 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144e6ae-b7e4-4d45-88db-ef1e5995a8a2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:38.867573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217b075a-84c2-4cc4-879f-cef808d1d43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:38.868424 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b075a-84c2-4cc4-879f-cef808d1d43a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:38.868711 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b075a-84c2-4cc4-879f-cef808d1d43a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:38.868992 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b075a-84c2-4cc4-879f-cef808d1d43a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:38.878911 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217b075a-84c2-4cc4-879f-cef808d1d43a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:22:38.887202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-217b075a-84c2-4cc4-879f-cef808d1d43a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:22:38.887202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:22:38.887202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:22:38.887202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:22:38.887202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:22:38.887202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:22:38.887202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:22:38.887202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:22:38.887202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:22:38.887202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:22:38.887202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:22:38.887202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:22:38.887202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:22:38.887202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:22:38.887202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:22:38.887202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:22:38.888284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:22:38.888284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:22:38.888284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:22:38.888284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:22:38.888284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:22:38.888284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:22:38.888284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:22:38.888284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:22:38.888284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:22:38.888284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:22:38.888284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:22:38.888284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:22:38.888284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:22:38.888284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:22:38.888284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:22:38.888284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:22:38.889333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:22:38.889333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:22:38.889333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:22:38.889333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:22:38.889333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:22:38.889333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:22:38.889333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:22:38.889333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:22:38.889333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:22:38.889333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:22:38.889333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:22:38.889333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:22:38.889333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 594/1188] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:22:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:22:38.894360 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a86a0df-f979-4766-bef7-1f38c76128fc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:38.989736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a86a0df-f979-4766-bef7-1f38c76128fc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:39.023634 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86dd1c-6dd5-4886-8b30-5b50b92e3320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:39.024287 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86dd1c-6dd5-4886-8b30-5b50b92e3320 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:39.024575 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86dd1c-6dd5-4886-8b30-5b50b92e3320 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:39.024854 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86dd1c-6dd5-4886-8b30-5b50b92e3320 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:39.025596 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86dd1c-6dd5-4886-8b30-5b50b92e3320 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:22:39.025908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86dd1c-6dd5-4886-8b30-5b50b92e3320 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:39.027303 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86dd1c-6dd5-4886-8b30-5b50b92e3320 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:39.035110 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 595/1189] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:22:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:39.041424 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217b075a-84c2-4cc4-879f-cef808d1d43a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:39.138149 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217b075a-84c2-4cc4-879f-cef808d1d43a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:39.170506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb68ce-e90a-4956-b20a-97c10f17fc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:39.171358 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb68ce-e90a-4956-b20a-97c10f17fc2b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:39.171641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb68ce-e90a-4956-b20a-97c10f17fc2b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:39.171919 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb68ce-e90a-4956-b20a-97c10f17fc2b None admin] PATH_INFO: `/v3/users/8d042d78ac9940f3b22756f17d59d95b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:39.173099 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceb68ce-e90a-4956-b20a-97c10f17fc2b None admin] RBAC: Authorizing `identity:get_user(user_id=8d042d78ac9940f3b22756f17d59d95b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:39.174561 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceb68ce-e90a-4956-b20a-97c10f17fc2b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:39.176168 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 595/1190] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:22:39 2022] GET /identity/v3/users/8d042d78ac9940f3b22756f17d59d95b => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:39.182198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86dd1c-6dd5-4886-8b30-5b50b92e3320 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:39.239342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:39.279426 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86dd1c-6dd5-4886-8b30-5b50b92e3320 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:39.311404 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e2f2b9-8ef4-4d2e-a91d-9012569d4e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:39.312246 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2f2b9-8ef4-4d2e-a91d-9012569d4e40 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:39.312530 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2f2b9-8ef4-4d2e-a91d-9012569d4e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:39.312804 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e2f2b9-8ef4-4d2e-a91d-9012569d4e40 None admin] PATH_INFO: `/v3/projects/c2dc36cebcaa480e99868c8e9815d1bd` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:39.313951 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e2f2b9-8ef4-4d2e-a91d-9012569d4e40 None admin] RBAC: Authorizing `identity:get_project(project_id=c2dc36cebcaa480e99868c8e9815d1bd)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:39.315387 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e2f2b9-8ef4-4d2e-a91d-9012569d4e40 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:39.317028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 596/1191] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:22:39 2022] GET /identity/v3/projects/c2dc36cebcaa480e99868c8e9815d1bd => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:39.323389 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb68ce-e90a-4956-b20a-97c10f17fc2b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:39.351418 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:39.420500 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb68ce-e90a-4956-b20a-97c10f17fc2b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:39.453244 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c57589-3fc2-4d85-a25b-883ca554f082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:39.454112 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c57589-3fc2-4d85-a25b-883ca554f082 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:39.454391 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c57589-3fc2-4d85-a25b-883ca554f082 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:39.454706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c57589-3fc2-4d85-a25b-883ca554f082 None admin] PATH_INFO: `/v3/projects/c2dc36cebcaa480e99868c8e9815d1bd/users/8d042d78ac9940f3b22756f17d59d95b/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:39.456608 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c57589-3fc2-4d85-a25b-883ca554f082 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2dc36cebcaa480e99868c8e9815d1bd, user_id=8d042d78ac9940f3b22756f17d59d95b, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:39.458024 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c57589-3fc2-4d85-a25b-883ca554f082 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:39.470621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 596/1192] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 12:22:39 2022] PUT /identity/v3/projects/c2dc36cebcaa480e99868c8e9815d1bd/users/8d042d78ac9940f3b22756f17d59d95b/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:22:41.463999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7867b07-db53-4fdb-9af0-8b4a42bacdf9 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:41.464534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7867b07-db53-4fdb-9af0-8b4a42bacdf9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:41.464628 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7867b07-db53-4fdb-9af0-8b4a42bacdf9 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:41.465917 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 597/1193] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:22:41.471151 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a9c7c-7256-40f5-8306-605a2e19274d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:41.471530 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a9c7c-7256-40f5-8306-605a2e19274d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:41.471716 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a9c7c-7256-40f5-8306-605a2e19274d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:41.488553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-fc7a9c7c-7256-40f5-8306-605a2e19274d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:22:41.489734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fc7a9c7c-7256-40f5-8306-605a2e19274d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:22:41.559794 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 597/1194] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:22:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:22:41.583431 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7867b07-db53-4fdb-9af0-8b4a42bacdf9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:41.684027 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7867b07-db53-4fdb-9af0-8b4a42bacdf9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:41.708460 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:41.716830 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16850e5-776b-4457-b22f-323f6c3278e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:41.717608 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16850e5-776b-4457-b22f-323f6c3278e8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:41.717892 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16850e5-776b-4457-b22f-323f6c3278e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:41.718168 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16850e5-776b-4457-b22f-323f6c3278e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:41.729027 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16850e5-776b-4457-b22f-323f6c3278e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:22:41.736507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a16850e5-776b-4457-b22f-323f6c3278e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:22:41.736507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:22:41.736507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:22:41.736507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:22:41.736507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:22:41.736507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:22:41.736507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:22:41.736507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:22:41.736507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:22:41.736507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:22:41.736507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:22:41.736507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:22:41.736507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:22:41.736507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:22:41.736507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:22:41.736507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:22:41.737625 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:22:41.737625 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:22:41.737625 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:22:41.737625 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:22:41.737625 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:22:41.737625 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:22:41.737625 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:22:41.737625 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:22:41.737625 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:22:41.737625 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:22:41.737625 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:22:41.737625 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:22:41.737625 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:22:41.737625 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:22:41.737625 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:22:41.737625 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:22:41.738559 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:22:41.738559 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:22:41.738559 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:22:41.738559 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:22:41.738559 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:22:41.738559 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:22:41.738559 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:22:41.738559 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:22:41.738559 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:22:41.738559 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:22:41.738559 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:22:41.738559 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:22:41.738559 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 598/1195] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:22:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:22:41.743944 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a9c7c-7256-40f5-8306-605a2e19274d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:41.761573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:41.840139 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a9c7c-7256-40f5-8306-605a2e19274d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:41.872729 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f267dd-12e7-45a3-a506-a0d85eaebc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:41.873383 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f267dd-12e7-45a3-a506-a0d85eaebc14 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:41.873667 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f267dd-12e7-45a3-a506-a0d85eaebc14 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:41.873945 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f267dd-12e7-45a3-a506-a0d85eaebc14 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:41.874714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f267dd-12e7-45a3-a506-a0d85eaebc14 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:22:41.875023 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f267dd-12e7-45a3-a506-a0d85eaebc14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:41.876378 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f267dd-12e7-45a3-a506-a0d85eaebc14 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:41.884892 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 598/1196] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:22:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:41.890647 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16850e5-776b-4457-b22f-323f6c3278e8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:41.988115 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16850e5-776b-4457-b22f-323f6c3278e8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:42.021152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab0188-87bd-437c-b634-081fec63abd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:42.022252 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab0188-87bd-437c-b634-081fec63abd6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:42.022558 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab0188-87bd-437c-b634-081fec63abd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:42.023421 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab0188-87bd-437c-b634-081fec63abd6 None admin] PATH_INFO: `/v3/users/8d042d78ac9940f3b22756f17d59d95b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:42.024588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab0188-87bd-437c-b634-081fec63abd6 None admin] RBAC: Authorizing `identity:get_user(user_id=8d042d78ac9940f3b22756f17d59d95b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:42.025998 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab0188-87bd-437c-b634-081fec63abd6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:42.027597 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 599/1197] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jun 29 12:22:41 2022] GET /identity/v3/users/8d042d78ac9940f3b22756f17d59d95b => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:42.033205 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f267dd-12e7-45a3-a506-a0d85eaebc14 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:42.115142 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:42.135288 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f267dd-12e7-45a3-a506-a0d85eaebc14 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:42.167498 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c17804-0936-49d9-a687-0f3f86761752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:42.168124 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c17804-0936-49d9-a687-0f3f86761752 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:42.168405 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c17804-0936-49d9-a687-0f3f86761752 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:42.168677 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c17804-0936-49d9-a687-0f3f86761752 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:42.232351 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c17804-0936-49d9-a687-0f3f86761752 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:42.233787 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c17804-0936-49d9-a687-0f3f86761752 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:42.310779 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:42.330305 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 599/1198] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:22:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 12:22:42.336282 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab0188-87bd-437c-b634-081fec63abd6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:42.391034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:42.434267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab0188-87bd-437c-b634-081fec63abd6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:42.467249 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857f8dbb-afa1-47d0-9019-483c15a7c627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:42.468095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f8dbb-afa1-47d0-9019-483c15a7c627 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:42.468384 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f8dbb-afa1-47d0-9019-483c15a7c627 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:42.468656 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857f8dbb-afa1-47d0-9019-483c15a7c627 None admin] PATH_INFO: `/v3/projects/c2dc36cebcaa480e99868c8e9815d1bd` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:42.469820 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f8dbb-afa1-47d0-9019-483c15a7c627 None admin] RBAC: Authorizing `identity:get_project(project_id=c2dc36cebcaa480e99868c8e9815d1bd)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:42.471261 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857f8dbb-afa1-47d0-9019-483c15a7c627 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:42.473090 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 600/1199] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jun 29 12:22:42 2022] GET /identity/v3/projects/c2dc36cebcaa480e99868c8e9815d1bd => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:22:42.479172 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c17804-0936-49d9-a687-0f3f86761752 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:22:42.574680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c17804-0936-49d9-a687-0f3f86761752 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:22:42.613075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82eae0-6118-445c-b6ef-ddbf253d6575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:22:42.613746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82eae0-6118-445c-b6ef-ddbf253d6575 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:22:42.614031 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82eae0-6118-445c-b6ef-ddbf253d6575 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:22:42.614310 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82eae0-6118-445c-b6ef-ddbf253d6575 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:22:42.615172 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b82eae0-6118-445c-b6ef-ddbf253d6575 None admin] RBAC: Adding query filter params (role.id=3a7f4bf556344407a6d9540c89d13641, scope.project.id=c2dc36cebcaa480e99868c8e9815d1bd, user.id=8d042d78ac9940f3b22756f17d59d95b) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:22:42.615486 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b82eae0-6118-445c-b6ef-ddbf253d6575 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:22:42.617469 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b82eae0-6118-445c-b6ef-ddbf253d6575 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:22:42.623196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 600/1200] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:22:42 2022] GET /identity/v3/role_assignments?user.id=8d042d78ac9940f3b22756f17d59d95b&scope.project.id=c2dc36cebcaa480e99868c8e9815d1bd&role.id=3a7f4bf556344407a6d9540c89d13641 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:22:52.463239 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=38 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:22:52.620657 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=78 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:38.000875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56321e-99eb-443b-b71e-dd4ed9e403be None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:38.001598 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56321e-99eb-443b-b71e-dd4ed9e403be None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:38.001598 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56321e-99eb-443b-b71e-dd4ed9e403be None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:38.002825 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 601/1201] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:23:38.012701 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5f3a8-efc7-49da-9cf7-94176f23191e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:38.012701 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5f3a8-efc7-49da-9cf7-94176f23191e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:38.012701 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5f3a8-efc7-49da-9cf7-94176f23191e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:38.033242 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-fff5f3a8-efc7-49da-9cf7-94176f23191e None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:23:38.034535 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fff5f3a8-efc7-49da-9cf7-94176f23191e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:23:38.138653 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 601/1202] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:23:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:23:38.164763 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56321e-99eb-443b-b71e-dd4ed9e403be None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:38.293673 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56321e-99eb-443b-b71e-dd4ed9e403be None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:38.343767 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a574c0-230a-4e45-bbd3-a9c4a04d760c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:38.343767 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a574c0-230a-4e45-bbd3-a9c4a04d760c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:38.343767 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a574c0-230a-4e45-bbd3-a9c4a04d760c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:38.344280 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a574c0-230a-4e45-bbd3-a9c4a04d760c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:38.359079 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a574c0-230a-4e45-bbd3-a9c4a04d760c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:38.360875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a574c0-230a-4e45-bbd3-a9c4a04d760c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:38.390112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-16a574c0-230a-4e45-bbd3-a9c4a04d760c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:23:38.390112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:23:38.390112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:23:38.390112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:23:38.390112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:23:38.390112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:23:38.390112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:23:38.390112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:23:38.390112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:23:38.390112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:23:38.390112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:23:38.390112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:23:38.390112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:23:38.390112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:23:38.390112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:23:38.391540 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:23:38.391540 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:23:38.391540 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:23:38.391540 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:23:38.391540 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:23:38.391540 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:23:38.391540 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:23:38.391540 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:23:38.391540 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:23:38.391540 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:23:38.391540 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:23:38.391540 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:23:38.391540 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:23:38.391540 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:23:38.391540 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:23:38.391540 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:23:38.391540 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:23:38.393072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:23:38.393072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:23:38.393072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:23:38.393072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:23:38.393072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 602/1203] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:23:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:23:38.399488 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff5f3a8-efc7-49da-9cf7-94176f23191e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:38.525588 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff5f3a8-efc7-49da-9cf7-94176f23191e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:38.565160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:38.572769 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedebbbf-1000-4369-804a-2f81d4516c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:38.573090 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedebbbf-1000-4369-804a-2f81d4516c28 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:38.573347 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedebbbf-1000-4369-804a-2f81d4516c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:38.573917 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedebbbf-1000-4369-804a-2f81d4516c28 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:38.574909 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedebbbf-1000-4369-804a-2f81d4516c28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:23:38.575246 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedebbbf-1000-4369-804a-2f81d4516c28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:38.577171 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedebbbf-1000-4369-804a-2f81d4516c28 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:38.591349 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 602/1204] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:23:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:23:38.597935 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a574c0-230a-4e45-bbd3-a9c4a04d760c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:38.666848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:38.742054 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a574c0-230a-4e45-bbd3-a9c4a04d760c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:38.788820 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd522629-cf49-45cd-8755-2075cdef2d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:38.789544 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd522629-cf49-45cd-8755-2075cdef2d57 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:38.789838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd522629-cf49-45cd-8755-2075cdef2d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:38.790324 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd522629-cf49-45cd-8755-2075cdef2d57 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:38.791066 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd522629-cf49-45cd-8755-2075cdef2d57 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:38.792696 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd522629-cf49-45cd-8755-2075cdef2d57 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:38.822993 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 603/1205] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jun 29 12:23:38 2022] POST /identity/v3/users => generated 251 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:23:41.273916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62513d-4237-421f-b581-58422b150d9a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:41.274479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62513d-4237-421f-b581-58422b150d9a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:41.274590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62513d-4237-421f-b581-58422b150d9a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:41.275982 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 603/1206] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:23:41.281471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85314f27-7224-4144-ac02-bda99dedd477 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:41.282241 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85314f27-7224-4144-ac02-bda99dedd477 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:41.282241 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85314f27-7224-4144-ac02-bda99dedd477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:41.300739 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-85314f27-7224-4144-ac02-bda99dedd477 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:23:41.301959 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-85314f27-7224-4144-ac02-bda99dedd477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:23:41.380203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 604/1207] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:23:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:23:41.403939 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62513d-4237-421f-b581-58422b150d9a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:41.524658 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62513d-4237-421f-b581-58422b150d9a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:41.565915 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068a1412-5742-46eb-a6f1-b74ea0388409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:41.566969 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a1412-5742-46eb-a6f1-b74ea0388409 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:41.567229 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a1412-5742-46eb-a6f1-b74ea0388409 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:41.567545 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a1412-5742-46eb-a6f1-b74ea0388409 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:41.580998 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068a1412-5742-46eb-a6f1-b74ea0388409 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:23:41.589573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-068a1412-5742-46eb-a6f1-b74ea0388409 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:23:41.589573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:23:41.589573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:23:41.589573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:23:41.589573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:23:41.589573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:23:41.589573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:23:41.589573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:23:41.589573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:23:41.589573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:23:41.589573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:23:41.589573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:23:41.589573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:23:41.589573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:23:41.589573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:23:41.589573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:23:41.590802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:23:41.590802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:23:41.590802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:23:41.590802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:23:41.590802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:23:41.590802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:23:41.590802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:23:41.590802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:23:41.590802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:23:41.590802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:23:41.590802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:23:41.590802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:23:41.590802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:23:41.590802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:23:41.590802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:23:41.590802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:23:41.591791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:23:41.591791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:23:41.591791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:23:41.591791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:23:41.591791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:23:41.591791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:23:41.591791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:23:41.591791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:23:41.591791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:23:41.591791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:23:41.591791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:23:41.591791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:23:41.591791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 604/1208] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 12:23:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:23:41.597008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85314f27-7224-4144-ac02-bda99dedd477 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:41.642058 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:41.717291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85314f27-7224-4144-ac02-bda99dedd477 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:41.757224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ac7b7-a1e5-4022-b9ba-413428ad90e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:41.757945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ac7b7-a1e5-4022-b9ba-413428ad90e9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:41.758244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ac7b7-a1e5-4022-b9ba-413428ad90e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:41.758653 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ac7b7-a1e5-4022-b9ba-413428ad90e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:41.759422 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ac7b7-a1e5-4022-b9ba-413428ad90e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:23:41.759863 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ac7b7-a1e5-4022-b9ba-413428ad90e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:41.761271 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ac7b7-a1e5-4022-b9ba-413428ad90e9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:41.770397 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 605/1209] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 12:23:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:23:41.776727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068a1412-5742-46eb-a6f1-b74ea0388409 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:41.833108 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:41.897803 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068a1412-5742-46eb-a6f1-b74ea0388409 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:41.939037 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd06bfe-116a-49fd-930d-d84e0a25fc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:41.939860 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd06bfe-116a-49fd-930d-d84e0a25fc34 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:41.940238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd06bfe-116a-49fd-930d-d84e0a25fc34 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:41.940417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd06bfe-116a-49fd-930d-d84e0a25fc34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:41.951994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd06bfe-116a-49fd-930d-d84e0a25fc34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:41.953573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd06bfe-116a-49fd-930d-d84e0a25fc34 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:41.973662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-1bd06bfe-116a-49fd-930d-d84e0a25fc34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:23:41.973662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:23:41.973662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:23:41.973662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:23:41.973662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:23:41.973662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:23:41.973662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:23:41.973662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:23:41.973662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:23:41.973662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:23:41.973662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:23:41.973662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:23:41.973662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:23:41.973662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:23:41.973662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:23:41.980509 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:23:41.980509 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:23:41.980509 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:23:41.980509 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:23:41.980509 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:23:41.980509 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:23:41.980509 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:23:41.980509 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:23:41.980509 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:23:41.980509 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:23:41.980509 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:23:41.980509 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:23:41.980509 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:23:41.980509 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:23:41.980509 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:23:41.980509 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:23:41.980509 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:23:41.981640 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:23:41.981640 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:23:41.981640 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:23:41.981640 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:23:41.981640 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 605/1210] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:23:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:23:41.981992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ac7b7-a1e5-4022-b9ba-413428ad90e9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:42.100957 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:42.104115 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ac7b7-a1e5-4022-b9ba-413428ad90e9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:42.143839 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd2a7f-3779-426e-be31-dca26b12207c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:42.144515 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd2a7f-3779-426e-be31-dca26b12207c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:42.144811 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd2a7f-3779-426e-be31-dca26b12207c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:42.145116 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd2a7f-3779-426e-be31-dca26b12207c None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:42.145814 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd2a7f-3779-426e-be31-dca26b12207c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:23:42.146279 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd2a7f-3779-426e-be31-dca26b12207c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:42.149126 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd2a7f-3779-426e-be31-dca26b12207c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:42.163286 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 606/1211] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:23:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:23:42.168994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd06bfe-116a-49fd-930d-d84e0a25fc34 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:42.293205 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd06bfe-116a-49fd-930d-d84e0a25fc34 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:42.294886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:42.335219 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505bf08-af0c-46bb-90a9-f6f676f2ec07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:42.336324 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505bf08-af0c-46bb-90a9-f6f676f2ec07 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:42.336564 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505bf08-af0c-46bb-90a9-f6f676f2ec07 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:42.336849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505bf08-af0c-46bb-90a9-f6f676f2ec07 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:42.345482 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2505bf08-af0c-46bb-90a9-f6f676f2ec07 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:42.347134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2505bf08-af0c-46bb-90a9-f6f676f2ec07 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:42.363078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-2505bf08-af0c-46bb-90a9-f6f676f2ec07 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 12:23:42.363078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:23:42.363078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:23:42.363078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:23:42.363078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:23:42.363078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:23:42.363078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:23:42.363078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:23:42.363078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:23:42.363078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:23:42.363078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:23:42.363078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:23:42.363078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:23:42.363078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:23:42.363078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:23:42.363078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:23:42.364180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:23:42.364180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:23:42.364180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:23:42.364180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:23:42.364180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:23:42.364180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:23:42.364180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:23:42.364180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:23:42.364180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:23:42.364180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:23:42.364180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:23:42.364180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:23:42.364180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:23:42.364180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:23:42.364180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:23:42.364180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:23:42.364180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:23:42.364180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:23:42.365735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:23:42.365735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:23:42.365735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:23:42.365735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:23:42.365735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:23:42.365735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:23:42.365735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:23:42.365735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:23:42.365735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 12:23:42.365735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:23:42.370465 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 606/1212] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:23:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:23:42.371362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd2a7f-3779-426e-be31-dca26b12207c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:42.503305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd2a7f-3779-426e-be31-dca26b12207c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:42.542676 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cba34b-42c8-4183-8e5b-570f33b4d66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:42.544094 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cba34b-42c8-4183-8e5b-570f33b4d66a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:42.544094 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cba34b-42c8-4183-8e5b-570f33b4d66a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:42.544094 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cba34b-42c8-4183-8e5b-570f33b4d66a None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:42.544742 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cba34b-42c8-4183-8e5b-570f33b4d66a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:23:42.545047 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cba34b-42c8-4183-8e5b-570f33b4d66a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:42.546549 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cba34b-42c8-4183-8e5b-570f33b4d66a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:42.569438 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 607/1213] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Jun 29 12:23:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:23:42.577464 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505bf08-af0c-46bb-90a9-f6f676f2ec07 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:42.701441 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505bf08-af0c-46bb-90a9-f6f676f2ec07 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:42.743421 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a020754-e237-4b16-a4d8-c35a6eadaee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:42.744109 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a020754-e237-4b16-a4d8-c35a6eadaee5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:42.744396 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a020754-e237-4b16-a4d8-c35a6eadaee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:42.744700 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a020754-e237-4b16-a4d8-c35a6eadaee5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:42.785892 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:42.832514 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a020754-e237-4b16-a4d8-c35a6eadaee5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:42.834190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a020754-e237-4b16-a4d8-c35a6eadaee5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:42.961515 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 607/1214] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:23:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 12:23:42.963128 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cba34b-42c8-4183-8e5b-570f33b4d66a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:42.993560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:43.094963 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cba34b-42c8-4183-8e5b-570f33b4d66a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:43.142522 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d01c6-b209-4897-8b79-1886da566982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:43.143409 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d01c6-b209-4897-8b79-1886da566982 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:43.143695 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d01c6-b209-4897-8b79-1886da566982 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:43.143972 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677d01c6-b209-4897-8b79-1886da566982 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:43.157432 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d01c6-b209-4897-8b79-1886da566982 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:43.158994 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677d01c6-b209-4897-8b79-1886da566982 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:43.185033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-677d01c6-b209-4897-8b79-1886da566982 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:23:43.185033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:23:43.185033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:23:43.185033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:23:43.185033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:23:43.185033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:23:43.185033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:23:43.185033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:23:43.185033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:23:43.185033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:23:43.185033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:23:43.185033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:23:43.185033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:23:43.185033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:23:43.185033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:23:43.186015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:23:43.186015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:23:43.186015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:23:43.186015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:23:43.186015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:23:43.186015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:23:43.186015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:23:43.186015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:23:43.186015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:23:43.186015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:23:43.186015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:23:43.186015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:23:43.186015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:23:43.186015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:23:43.186015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:23:43.186015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:23:43.186015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:23:43.187000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:23:43.187000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:23:43.187000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:23:43.187000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:23:43.191115 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 608/1215] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:23:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:23:43.192684 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a020754-e237-4b16-a4d8-c35a6eadaee5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:43.264999 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:43.319554 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a020754-e237-4b16-a4d8-c35a6eadaee5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:43.360529 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b396b68-a6fb-42f7-bbbd-ed82e1f12f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:43.360529 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b396b68-a6fb-42f7-bbbd-ed82e1f12f6f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:43.360529 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b396b68-a6fb-42f7-bbbd-ed82e1f12f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:43.360529 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b396b68-a6fb-42f7-bbbd-ed82e1f12f6f None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:43.360979 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b396b68-a6fb-42f7-bbbd-ed82e1f12f6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:23:43.360979 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b396b68-a6fb-42f7-bbbd-ed82e1f12f6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:43.362459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b396b68-a6fb-42f7-bbbd-ed82e1f12f6f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:43.374992 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 608/1216] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:23:43.382358 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d01c6-b209-4897-8b79-1886da566982 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:43.473107 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:43.510916 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677d01c6-b209-4897-8b79-1886da566982 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:43.556309 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae690d64-6fe4-4b76-aa77-60848fd71182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:43.557015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae690d64-6fe4-4b76-aa77-60848fd71182 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:43.557302 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae690d64-6fe4-4b76-aa77-60848fd71182 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:43.557577 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae690d64-6fe4-4b76-aa77-60848fd71182 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:43.567549 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae690d64-6fe4-4b76-aa77-60848fd71182 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:43.569432 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae690d64-6fe4-4b76-aa77-60848fd71182 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:43.586038 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ae690d64-6fe4-4b76-aa77-60848fd71182 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:23:43.586038 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:23:43.586038 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:23:43.586038 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:23:43.586038 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:23:43.586038 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:23:43.586038 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:23:43.586038 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:23:43.586038 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:23:43.586038 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:23:43.586038 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:23:43.586038 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:23:43.586038 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:23:43.586038 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:23:43.586038 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:23:43.587107 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:23:43.587107 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:23:43.587107 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:23:43.587107 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:23:43.587107 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:23:43.587107 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:23:43.587107 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:23:43.587107 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:23:43.587107 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:23:43.587107 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:23:43.587107 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:23:43.587107 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:23:43.587107 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:23:43.587107 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:23:43.587107 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:23:43.587107 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:23:43.587107 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:23:43.588002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:23:43.588002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:23:43.588002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:23:43.588002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:23:43.588002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:23:43.588002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:23:43.588002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:23:43.588002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:23:43.588002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 609/1217] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:23:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:23:43.594035 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b396b68-a6fb-42f7-bbbd-ed82e1f12f6f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:43.714370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b396b68-a6fb-42f7-bbbd-ed82e1f12f6f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:43.741517 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:43.762720 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce699e88-c401-45ad-b9cc-5d7a77b9dcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:43.763397 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce699e88-c401-45ad-b9cc-5d7a77b9dcd4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:43.763753 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce699e88-c401-45ad-b9cc-5d7a77b9dcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:43.763978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce699e88-c401-45ad-b9cc-5d7a77b9dcd4 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:43.765049 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce699e88-c401-45ad-b9cc-5d7a77b9dcd4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:23:43.765366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce699e88-c401-45ad-b9cc-5d7a77b9dcd4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:43.766930 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce699e88-c401-45ad-b9cc-5d7a77b9dcd4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:43.785789 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 609/1218] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:23:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:23:43.788342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae690d64-6fe4-4b76-aa77-60848fd71182 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:43.899173 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae690d64-6fe4-4b76-aa77-60848fd71182 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:43.931488 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:43.942590 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7183d632-eb90-44e7-8e11-0870757b4f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:43.943352 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183d632-eb90-44e7-8e11-0870757b4f72 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:43.943661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183d632-eb90-44e7-8e11-0870757b4f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:43.943908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7183d632-eb90-44e7-8e11-0870757b4f72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:43.944750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7183d632-eb90-44e7-8e11-0870757b4f72 None admin] RBAC: Adding query filter params (role.id=8883c151055d4ba493f514ce1bb62e78, scope.project.id=f043ade9e4f748d4adc010ffb0a47a46, user.id=c98135cacb9d4f22846ad33500b9bd1f) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:23:43.945060 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7183d632-eb90-44e7-8e11-0870757b4f72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:43.946517 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7183d632-eb90-44e7-8e11-0870757b4f72 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:43.952513 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 610/1219] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:23:43 2022] GET /identity/v3/role_assignments?user.id=c98135cacb9d4f22846ad33500b9bd1f&scope.project.id=f043ade9e4f748d4adc010ffb0a47a46&role.id=8883c151055d4ba493f514ce1bb62e78 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:23:46.007125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4995f-1e25-4053-aa1d-9c5f8ad23164 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:46.007652 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4995f-1e25-4053-aa1d-9c5f8ad23164 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:46.007652 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d4995f-1e25-4053-aa1d-9c5f8ad23164 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:46.009032 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 610/1220] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:23:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:23:46.014252 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57539f85-5cac-4710-988b-3913416eb1b7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:46.014696 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57539f85-5cac-4710-988b-3913416eb1b7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:46.014870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57539f85-5cac-4710-988b-3913416eb1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:46.032114 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-57539f85-5cac-4710-988b-3913416eb1b7 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:23:46.033308 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-57539f85-5cac-4710-988b-3913416eb1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:23:46.107699 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 611/1221] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:23:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:23:46.131297 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4995f-1e25-4053-aa1d-9c5f8ad23164 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:46.233442 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d4995f-1e25-4053-aa1d-9c5f8ad23164 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:46.266959 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602bb50-fa7b-46bd-92a0-1c2478175ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:46.267672 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602bb50-fa7b-46bd-92a0-1c2478175ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:46.267954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602bb50-fa7b-46bd-92a0-1c2478175ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:46.268232 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602bb50-fa7b-46bd-92a0-1c2478175ce9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:46.284217 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602bb50-fa7b-46bd-92a0-1c2478175ce9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:23:46.292630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9602bb50-fa7b-46bd-92a0-1c2478175ce9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:23:46.292630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:23:46.292630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:23:46.292630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:23:46.292630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:23:46.292630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:23:46.292630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:23:46.292630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:23:46.292630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:23:46.292630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:23:46.292630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:23:46.292630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:23:46.292630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:23:46.292630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:23:46.292630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:23:46.292630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:23:46.293748 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:23:46.293748 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:23:46.293748 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:23:46.293748 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:23:46.293748 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:23:46.293748 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:23:46.293748 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:23:46.293748 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:23:46.293748 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:23:46.293748 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:23:46.293748 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:23:46.293748 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:23:46.293748 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:23:46.293748 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:23:46.293748 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:23:46.293748 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:23:46.294684 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:23:46.294684 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:23:46.294684 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:23:46.294684 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:23:46.294684 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:23:46.294684 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:23:46.294684 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:23:46.294684 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:23:46.294684 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:23:46.294684 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:23:46.294684 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:23:46.294684 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:23:46.294684 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 611/1222] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 12:23:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:23:46.300117 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57539f85-5cac-4710-988b-3913416eb1b7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:46.409257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57539f85-5cac-4710-988b-3913416eb1b7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:46.434120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:46.445995 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a30d4-78b0-4381-9852-312782f4775b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:46.446789 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a30d4-78b0-4381-9852-312782f4775b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:46.447049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a30d4-78b0-4381-9852-312782f4775b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:46.447346 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a30d4-78b0-4381-9852-312782f4775b None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:46.448173 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a30d4-78b0-4381-9852-312782f4775b None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:23:46.448482 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a30d4-78b0-4381-9852-312782f4775b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:46.449896 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a30d4-78b0-4381-9852-312782f4775b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:46.463289 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 612/1223] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 12:23:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:23:46.464593 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602bb50-fa7b-46bd-92a0-1c2478175ce9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:46.510461 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:46.569182 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602bb50-fa7b-46bd-92a0-1c2478175ce9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:46.604883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2bfce-b8c7-4786-94bd-2ea7055fe307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:46.605724 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2bfce-b8c7-4786-94bd-2ea7055fe307 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:46.605965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2bfce-b8c7-4786-94bd-2ea7055fe307 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:46.606260 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2bfce-b8c7-4786-94bd-2ea7055fe307 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:46.616830 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f2bfce-b8c7-4786-94bd-2ea7055fe307 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:46.618353 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f2bfce-b8c7-4786-94bd-2ea7055fe307 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:46.636065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-83f2bfce-b8c7-4786-94bd-2ea7055fe307 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:23:46.636065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:23:46.636065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:23:46.636065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:23:46.636065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:23:46.636065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:23:46.636065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:23:46.636065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:23:46.636065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:23:46.636065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:23:46.636065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:23:46.636065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:23:46.636065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:23:46.636065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:23:46.636065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:23:46.637243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:23:46.637243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:23:46.637243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:23:46.637243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:23:46.637243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:23:46.637243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:23:46.637243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:23:46.637243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:23:46.637243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:23:46.637243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:23:46.637243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:23:46.637243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:23:46.637243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:23:46.637243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:23:46.637243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:23:46.637243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:23:46.637243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:23:46.638177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:23:46.638177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:23:46.638177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:23:46.638177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:23:46.638177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 612/1224] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:23:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:23:46.643438 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a30d4-78b0-4381-9852-312782f4775b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:46.756015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a30d4-78b0-4381-9852-312782f4775b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:46.798119 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cef85a-300c-4ddc-85bd-0b942ed14467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:46.798837 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cef85a-300c-4ddc-85bd-0b942ed14467 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:46.799166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cef85a-300c-4ddc-85bd-0b942ed14467 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:46.799661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cef85a-300c-4ddc-85bd-0b942ed14467 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:46.800506 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cef85a-300c-4ddc-85bd-0b942ed14467 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:23:46.800833 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cef85a-300c-4ddc-85bd-0b942ed14467 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:46.803291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cef85a-300c-4ddc-85bd-0b942ed14467 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:46.819809 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 613/1225] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:23:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:23:46.825543 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2bfce-b8c7-4786-94bd-2ea7055fe307 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:46.931669 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:46.939233 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2bfce-b8c7-4786-94bd-2ea7055fe307 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:46.976744 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d977fc1-8099-40f8-a24d-5c83689957af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:46.977659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d977fc1-8099-40f8-a24d-5c83689957af None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:46.977954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d977fc1-8099-40f8-a24d-5c83689957af None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:46.978224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d977fc1-8099-40f8-a24d-5c83689957af None admin] PATH_INFO: `/v3/users/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:46.987291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d977fc1-8099-40f8-a24d-5c83689957af None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:46.988991 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d977fc1-8099-40f8-a24d-5c83689957af None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:47.004366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-8d977fc1-8099-40f8-a24d-5c83689957af None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 12:23:47.004366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:23:47.004366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:23:47.004366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:23:47.004366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:23:47.004366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:23:47.004366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:23:47.004366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:23:47.004366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:23:47.004366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:23:47.004366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:23:47.004366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:23:47.004366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:23:47.004366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:23:47.004366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:23:47.004366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:23:47.006349 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:23:47.006349 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:23:47.006349 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:23:47.006349 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:23:47.006349 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:23:47.006349 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:23:47.006349 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:23:47.006349 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:23:47.006349 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:23:47.006349 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:23:47.006349 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:23:47.006349 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:23:47.006349 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:23:47.006349 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:23:47.006349 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:23:47.006349 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:23:47.006349 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:23:47.006349 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:23:47.007819 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:23:47.007819 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:23:47.007819 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:23:47.007819 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:23:47.007819 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:23:47.007819 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:23:47.007819 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:23:47.007819 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:23:47.007819 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 12:23:47.007819 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:23:47.007819 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 613/1226] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:23:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:23:47.012179 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cef85a-300c-4ddc-85bd-0b942ed14467 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:47.063907 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:47.127546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cef85a-300c-4ddc-85bd-0b942ed14467 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:47.165716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1cc73b-951d-4a47-a079-5080ff3146f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:47.166404 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cc73b-951d-4a47-a079-5080ff3146f6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:47.166724 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cc73b-951d-4a47-a079-5080ff3146f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:47.167006 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1cc73b-951d-4a47-a079-5080ff3146f6 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:47.167936 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1cc73b-951d-4a47-a079-5080ff3146f6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:23:47.168258 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1cc73b-951d-4a47-a079-5080ff3146f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:47.169819 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1cc73b-951d-4a47-a079-5080ff3146f6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:47.189776 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 614/1227] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Jun 29 12:23:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:23:47.195996 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d977fc1-8099-40f8-a24d-5c83689957af None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:47.311609 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d977fc1-8099-40f8-a24d-5c83689957af None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:47.346090 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07c398-8258-4173-b58f-8534b33cb8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:47.346951 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07c398-8258-4173-b58f-8534b33cb8d9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:47.347222 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07c398-8258-4173-b58f-8534b33cb8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:47.347545 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07c398-8258-4173-b58f-8534b33cb8d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:47.359362 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d07c398-8258-4173-b58f-8534b33cb8d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:47.360830 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d07c398-8258-4173-b58f-8534b33cb8d9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:47.382553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-2d07c398-8258-4173-b58f-8534b33cb8d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:23:47.382553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:23:47.382553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:23:47.382553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:23:47.382553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:23:47.382553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:23:47.382553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:23:47.382553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:23:47.382553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:23:47.382553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:23:47.382553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:23:47.382553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:23:47.382553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:23:47.382553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:23:47.382553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:23:47.384775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:23:47.384775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:23:47.384775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:23:47.384775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:23:47.384775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:23:47.384775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:23:47.384775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:23:47.384775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:23:47.384775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:23:47.384775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:23:47.384775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:23:47.384775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:23:47.384775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:23:47.384775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:23:47.384775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:23:47.384775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:23:47.384775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:23:47.386029 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:23:47.386029 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:23:47.386029 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:23:47.386029 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:23:47.386029 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 614/1228] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:23:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:23:47.390461 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1cc73b-951d-4a47-a079-5080ff3146f6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:47.506449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1cc73b-951d-4a47-a079-5080ff3146f6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:47.508945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:47.548144 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9f088-d63f-4f6c-9e22-83fa6aee0204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:47.549063 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9f088-d63f-4f6c-9e22-83fa6aee0204 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:47.549430 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9f088-d63f-4f6c-9e22-83fa6aee0204 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:47.549734 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b9f088-d63f-4f6c-9e22-83fa6aee0204 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:47.550571 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9f088-d63f-4f6c-9e22-83fa6aee0204 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:23:47.550891 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9f088-d63f-4f6c-9e22-83fa6aee0204 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:47.552310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b9f088-d63f-4f6c-9e22-83fa6aee0204 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:47.564370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 615/1229] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:23:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:23:47.571114 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07c398-8258-4173-b58f-8534b33cb8d9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:47.581188 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:47.684084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07c398-8258-4173-b58f-8534b33cb8d9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:47.721203 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb51a7-08cf-490c-93db-866d2e2eb68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:47.722041 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb51a7-08cf-490c-93db-866d2e2eb68d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:47.722324 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb51a7-08cf-490c-93db-866d2e2eb68d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:47.722628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cb51a7-08cf-490c-93db-866d2e2eb68d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:47.730091 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cb51a7-08cf-490c-93db-866d2e2eb68d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:47.731569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cb51a7-08cf-490c-93db-866d2e2eb68d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:47.746549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-39cb51a7-08cf-490c-93db-866d2e2eb68d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:23:47.746549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:23:47.746549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:23:47.746549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:23:47.746549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:23:47.746549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:23:47.746549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:23:47.746549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:23:47.746549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:23:47.746549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:23:47.746549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:23:47.746549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:23:47.746549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:23:47.746549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:23:47.746549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:23:47.747783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:23:47.747783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:23:47.747783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:23:47.747783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:23:47.747783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:23:47.747783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:23:47.747783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:23:47.747783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:23:47.747783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:23:47.747783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:23:47.747783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:23:47.747783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:23:47.747783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:23:47.747783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:23:47.747783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:23:47.747783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:23:47.747783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:23:47.749005 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:23:47.749005 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:23:47.749005 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:23:47.749005 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:23:47.749005 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:23:47.749005 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:23:47.749005 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:23:47.749005 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:23:47.752908 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 615/1230] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:23:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:23:47.753842 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9f088-d63f-4f6c-9e22-83fa6aee0204 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:47.867596 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b9f088-d63f-4f6c-9e22-83fa6aee0204 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:47.906707 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c801a5b0-5af5-4175-91f7-a357339e8456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:47.907400 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801a5b0-5af5-4175-91f7-a357339e8456 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:47.907671 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801a5b0-5af5-4175-91f7-a357339e8456 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:47.907989 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c801a5b0-5af5-4175-91f7-a357339e8456 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:47.908868 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c801a5b0-5af5-4175-91f7-a357339e8456 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:23:47.909204 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c801a5b0-5af5-4175-91f7-a357339e8456 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:47.910861 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c801a5b0-5af5-4175-91f7-a357339e8456 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:47.928294 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 616/1231] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:23:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:23:47.934364 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb51a7-08cf-490c-93db-866d2e2eb68d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:48.003712 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:48.041669 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cb51a7-08cf-490c-93db-866d2e2eb68d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:48.080694 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611d11b-e683-4280-b744-59f51c8e6828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:48.081849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611d11b-e683-4280-b744-59f51c8e6828 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:48.082177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611d11b-e683-4280-b744-59f51c8e6828 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:48.082569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611d11b-e683-4280-b744-59f51c8e6828 None admin] PATH_INFO: `/v3/projects/f043ade9e4f748d4adc010ffb0a47a46/users/c98135cacb9d4f22846ad33500b9bd1f/roles/8883c151055d4ba493f514ce1bb62e78` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:48.103326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7611d11b-e683-4280-b744-59f51c8e6828 None admin] RBAC: Authorizing `identity:create_grant(project_id=f043ade9e4f748d4adc010ffb0a47a46, user_id=c98135cacb9d4f22846ad33500b9bd1f, role_id=8883c151055d4ba493f514ce1bb62e78)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:48.104782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7611d11b-e683-4280-b744-59f51c8e6828 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:48.119397 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 616/1232] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 12:23:47 2022] PUT /identity/v3/projects/f043ade9e4f748d4adc010ffb0a47a46/users/c98135cacb9d4f22846ad33500b9bd1f/roles/8883c151055d4ba493f514ce1bb62e78 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:23:50.308343 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4cafd-e9c2-4239-8b6b-3f920d3e8510 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:50.308982 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4cafd-e9c2-4239-8b6b-3f920d3e8510 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:50.308982 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4cafd-e9c2-4239-8b6b-3f920d3e8510 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:50.310239 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 617/1233] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:23:50.316166 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64286f-7875-4e2f-a905-5b1103809d6d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:50.316638 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64286f-7875-4e2f-a905-5b1103809d6d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:50.316779 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e64286f-7875-4e2f-a905-5b1103809d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:50.333932 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6e64286f-7875-4e2f-a905-5b1103809d6d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:23:50.335411 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6e64286f-7875-4e2f-a905-5b1103809d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:23:50.413631 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 617/1234] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:23:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:23:50.436822 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4cafd-e9c2-4239-8b6b-3f920d3e8510 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:50.464613 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:50.551886 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4cafd-e9c2-4239-8b6b-3f920d3e8510 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:50.589019 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6853b9-04d9-4577-8e32-4db8e503ecb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:50.589721 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6853b9-04d9-4577-8e32-4db8e503ecb2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:50.590004 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6853b9-04d9-4577-8e32-4db8e503ecb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:50.590353 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6853b9-04d9-4577-8e32-4db8e503ecb2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:50.600602 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6853b9-04d9-4577-8e32-4db8e503ecb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:23:50.608351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ab6853b9-04d9-4577-8e32-4db8e503ecb2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:23:50.608351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:23:50.608351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:23:50.608351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:23:50.608351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:23:50.608351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:23:50.608351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:23:50.608351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:23:50.608351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:23:50.608351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:23:50.608351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:23:50.608351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:23:50.608351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:23:50.608351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:23:50.608351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:23:50.608351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:23:50.609470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:23:50.609470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:23:50.609470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:23:50.609470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:23:50.609470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:23:50.609470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:23:50.609470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:23:50.609470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:23:50.609470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:23:50.609470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:23:50.609470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:23:50.609470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:23:50.609470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:23:50.609470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:23:50.609470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:23:50.609470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:23:50.610684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:23:50.610684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:23:50.610684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:23:50.610684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:23:50.610684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:23:50.610684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:23:50.610684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:23:50.610684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:23:50.610684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:23:50.610684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:23:50.610684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 29 12:23:50.610684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:23:50.610684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 618/1235] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jun 29 12:23:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:23:50.618323 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e64286f-7875-4e2f-a905-5b1103809d6d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:50.666462 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:50.731529 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e64286f-7875-4e2f-a905-5b1103809d6d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:50.767745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd8b38-8cdc-49d0-a10e-9082f9e55666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:50.768465 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd8b38-8cdc-49d0-a10e-9082f9e55666 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:50.768746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd8b38-8cdc-49d0-a10e-9082f9e55666 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:50.769078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd8b38-8cdc-49d0-a10e-9082f9e55666 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:50.769876 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecd8b38-8cdc-49d0-a10e-9082f9e55666 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:23:50.770159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecd8b38-8cdc-49d0-a10e-9082f9e55666 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:50.771763 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecd8b38-8cdc-49d0-a10e-9082f9e55666 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:50.782214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 618/1236] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jun 29 12:23:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:23:50.788178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6853b9-04d9-4577-8e32-4db8e503ecb2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:50.883806 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:50.901410 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6853b9-04d9-4577-8e32-4db8e503ecb2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:50.936756 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc75019-0d8a-495f-a9c8-ff01b35897eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:50.937545 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc75019-0d8a-495f-a9c8-ff01b35897eb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:50.937826 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc75019-0d8a-495f-a9c8-ff01b35897eb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:50.938100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc75019-0d8a-495f-a9c8-ff01b35897eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:50.949531 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc75019-0d8a-495f-a9c8-ff01b35897eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:50.951214 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc75019-0d8a-495f-a9c8-ff01b35897eb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:50.969095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0dc75019-0d8a-495f-a9c8-ff01b35897eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:23:50.969095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:23:50.969095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:23:50.969095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:23:50.969095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:23:50.969095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:23:50.969095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:23:50.969095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:23:50.969095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:23:50.969095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:23:50.969095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:23:50.969095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:23:50.969095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:23:50.969095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:23:50.969095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:23:50.970140 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:23:50.970140 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:23:50.970140 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:23:50.970140 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:23:50.970140 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:23:50.970140 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:23:50.970140 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:23:50.970140 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:23:50.970140 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:23:50.970140 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:23:50.970140 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:23:50.970140 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:23:50.970140 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:23:50.970140 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:23:50.970140 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:23:50.970140 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:23:50.970140 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:23:50.971103 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:23:50.971103 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:23:50.971103 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:23:50.971103 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:23:50.971103 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 619/1237] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:23:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:23:50.977258 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd8b38-8cdc-49d0-a10e-9082f9e55666 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:51.091567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd8b38-8cdc-49d0-a10e-9082f9e55666 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:51.103456 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:51.133701 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4295693b-545e-4337-b6bf-bb9d57fda3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:51.134475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295693b-545e-4337-b6bf-bb9d57fda3a9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:51.134782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295693b-545e-4337-b6bf-bb9d57fda3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:51.135094 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295693b-545e-4337-b6bf-bb9d57fda3a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:51.135964 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4295693b-545e-4337-b6bf-bb9d57fda3a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:23:51.136376 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4295693b-545e-4337-b6bf-bb9d57fda3a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:51.138206 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4295693b-545e-4337-b6bf-bb9d57fda3a9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:51.154144 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 619/1238] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:23:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:23:51.159956 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc75019-0d8a-495f-a9c8-ff01b35897eb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:51.271995 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc75019-0d8a-495f-a9c8-ff01b35897eb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:51.313928 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d46e6-baf2-4e54-84ad-b9441bac7e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:51.314905 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d46e6-baf2-4e54-84ad-b9441bac7e65 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:51.315193 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d46e6-baf2-4e54-84ad-b9441bac7e65 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:51.315474 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55d46e6-baf2-4e54-84ad-b9441bac7e65 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:51.319959 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:51.326040 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d46e6-baf2-4e54-84ad-b9441bac7e65 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:51.327808 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55d46e6-baf2-4e54-84ad-b9441bac7e65 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:51.342628 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a55d46e6-baf2-4e54-84ad-b9441bac7e65 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 12:23:51.342628 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:23:51.342628 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:23:51.342628 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:23:51.342628 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:23:51.342628 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:23:51.342628 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:23:51.342628 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:23:51.342628 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:23:51.342628 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:23:51.342628 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:23:51.342628 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:23:51.342628 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:23:51.342628 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:23:51.342628 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:23:51.342628 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:23:51.343607 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:23:51.343607 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:23:51.343607 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:23:51.343607 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:23:51.343607 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:23:51.343607 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:23:51.343607 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:23:51.343607 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:23:51.343607 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:23:51.343607 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:23:51.343607 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:23:51.343607 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:23:51.343607 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:23:51.343607 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:23:51.343607 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:23:51.343607 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:23:51.343607 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:23:51.343607 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:23:51.344539 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:23:51.344539 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:23:51.344539 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:23:51.344539 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:23:51.344539 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:23:51.344539 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:23:51.344539 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:23:51.344539 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:23:51.344539 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 12:23:51.344539 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:23:51.344539 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 620/1239] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:23:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:23:51.351099 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4295693b-545e-4337-b6bf-bb9d57fda3a9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:51.469384 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4295693b-545e-4337-b6bf-bb9d57fda3a9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:51.507698 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ae699-c14b-4f96-84d5-a09a11e65d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:51.508377 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ae699-c14b-4f96-84d5-a09a11e65d95 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:51.508656 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ae699-c14b-4f96-84d5-a09a11e65d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:51.508931 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ae699-c14b-4f96-84d5-a09a11e65d95 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:51.509766 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ae699-c14b-4f96-84d5-a09a11e65d95 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:23:51.510079 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ae699-c14b-4f96-84d5-a09a11e65d95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:51.511532 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ae699-c14b-4f96-84d5-a09a11e65d95 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:51.529606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 620/1240] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Jun 29 12:23:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:23:51.535420 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d46e6-baf2-4e54-84ad-b9441bac7e65 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:51.644981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55d46e6-baf2-4e54-84ad-b9441bac7e65 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:51.687552 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35853e-2518-4875-80b1-c802057d5173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:51.688288 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35853e-2518-4875-80b1-c802057d5173 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:51.688613 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35853e-2518-4875-80b1-c802057d5173 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:51.688961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35853e-2518-4875-80b1-c802057d5173 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:51.756968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:51.764150 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e35853e-2518-4875-80b1-c802057d5173 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:51.765637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e35853e-2518-4875-80b1-c802057d5173 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:51.870537 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 621/1241] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:23:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 12:23:51.876786 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ae699-c14b-4f96-84d5-a09a11e65d95 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:51.911234 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:51.999451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ae699-c14b-4f96-84d5-a09a11e65d95 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:52.036226 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2610b-7ff7-493d-9f28-96f7281abee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:52.037045 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2610b-7ff7-493d-9f28-96f7281abee8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:52.037448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2610b-7ff7-493d-9f28-96f7281abee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:52.037741 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2610b-7ff7-493d-9f28-96f7281abee8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:52.048448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2610b-7ff7-493d-9f28-96f7281abee8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:52.049905 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc2610b-7ff7-493d-9f28-96f7281abee8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:52.070581 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6cc2610b-7ff7-493d-9f28-96f7281abee8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:23:52.070581 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:23:52.070581 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:23:52.070581 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:23:52.070581 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:23:52.070581 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:23:52.070581 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:23:52.070581 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:23:52.070581 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:23:52.070581 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:23:52.070581 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:23:52.070581 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:23:52.070581 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:23:52.070581 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:23:52.070581 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:23:52.071745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:23:52.071745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:23:52.071745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:23:52.071745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:23:52.071745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:23:52.071745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:23:52.071745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:23:52.071745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:23:52.071745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:23:52.071745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:23:52.071745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:23:52.071745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:23:52.071745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:23:52.071745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:23:52.071745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:23:52.071745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:23:52.071745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:23:52.072991 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:23:52.072991 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:23:52.072991 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:23:52.072991 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:23:52.072991 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 621/1242] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:23:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:23:52.077906 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35853e-2518-4875-80b1-c802057d5173 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:52.203707 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:52.211571 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35853e-2518-4875-80b1-c802057d5173 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:52.252420 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97772c-4fdb-479d-bdea-9126e00c95e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:52.253083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97772c-4fdb-479d-bdea-9126e00c95e7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:52.253387 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97772c-4fdb-479d-bdea-9126e00c95e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:52.253648 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97772c-4fdb-479d-bdea-9126e00c95e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:52.254371 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97772c-4fdb-479d-bdea-9126e00c95e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:23:52.254711 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97772c-4fdb-479d-bdea-9126e00c95e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:52.256810 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97772c-4fdb-479d-bdea-9126e00c95e7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:52.268500 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 622/1243] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:23:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:23:52.274799 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2610b-7ff7-493d-9f28-96f7281abee8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:52.360155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:52.395277 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2610b-7ff7-493d-9f28-96f7281abee8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:52.440037 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d5956-6d42-443c-ad85-8aacc2861a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:52.440808 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d5956-6d42-443c-ad85-8aacc2861a9c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:52.441095 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d5956-6d42-443c-ad85-8aacc2861a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:52.441418 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d5956-6d42-443c-ad85-8aacc2861a9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:52.449877 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d5956-6d42-443c-ad85-8aacc2861a9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:52.451455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d5956-6d42-443c-ad85-8aacc2861a9c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:52.467697 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-029d5956-6d42-443c-ad85-8aacc2861a9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:23:52.467697 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:23:52.467697 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:23:52.467697 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:23:52.467697 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:23:52.467697 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:23:52.467697 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:23:52.467697 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:23:52.467697 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:23:52.467697 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:23:52.467697 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:23:52.467697 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:23:52.467697 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:23:52.467697 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:23:52.467697 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:23:52.468775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:23:52.468775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:23:52.468775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:23:52.468775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:23:52.468775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:23:52.468775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:23:52.468775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:23:52.468775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:23:52.468775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:23:52.468775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:23:52.468775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:23:52.468775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:23:52.468775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:23:52.468775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:23:52.468775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:23:52.468775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:23:52.468775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:23:52.469694 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:23:52.469694 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:23:52.469694 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:23:52.469694 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:23:52.469694 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:23:52.469694 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:23:52.469694 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:23:52.469694 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:23:52.470593 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 622/1244] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:23:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:23:52.476032 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97772c-4fdb-479d-bdea-9126e00c95e7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:52.597414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97772c-4fdb-479d-bdea-9126e00c95e7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:52.632283 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b074e32-8a9e-403b-86a9-e19ccfd98e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:52.632897 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b074e32-8a9e-403b-86a9-e19ccfd98e33 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:52.633184 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b074e32-8a9e-403b-86a9-e19ccfd98e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:52.633462 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b074e32-8a9e-403b-86a9-e19ccfd98e33 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:52.634252 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b074e32-8a9e-403b-86a9-e19ccfd98e33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:23:52.634589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b074e32-8a9e-403b-86a9-e19ccfd98e33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:52.636019 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b074e32-8a9e-403b-86a9-e19ccfd98e33 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:52.649121 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 623/1245] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:23:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:23:52.655485 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d5956-6d42-443c-ad85-8aacc2861a9c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:52.773202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d5956-6d42-443c-ad85-8aacc2861a9c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:52.808959 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66d2b9-fc46-4e2c-933e-13b0219819d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:52.809620 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66d2b9-fc46-4e2c-933e-13b0219819d3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:52.809905 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66d2b9-fc46-4e2c-933e-13b0219819d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:52.810188 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66d2b9-fc46-4e2c-933e-13b0219819d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:52.811054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66d2b9-fc46-4e2c-933e-13b0219819d3 None admin] RBAC: Adding query filter params (role.id=8883c151055d4ba493f514ce1bb62e78, scope.project.id=f043ade9e4f748d4adc010ffb0a47a46, user.id=c98135cacb9d4f22846ad33500b9bd1f) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:23:52.811364 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66d2b9-fc46-4e2c-933e-13b0219819d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:52.812885 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66d2b9-fc46-4e2c-933e-13b0219819d3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:52.820128 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 623/1246] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:23:52 2022] GET /identity/v3/role_assignments?user.id=c98135cacb9d4f22846ad33500b9bd1f&scope.project.id=f043ade9e4f748d4adc010ffb0a47a46&role.id=8883c151055d4ba493f514ce1bb62e78 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:23:55.134373 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168bb70-408a-487a-a318-c0a89a6fc787 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:55.134981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168bb70-408a-487a-a318-c0a89a6fc787 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:55.135312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5168bb70-408a-487a-a318-c0a89a6fc787 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:55.137449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 624/1247] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:23:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:23:55.144029 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2c606-3a4a-4e17-be1d-a9ee173a5dba None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:55.144567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2c606-3a4a-4e17-be1d-a9ee173a5dba None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:55.144666 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb2c606-3a4a-4e17-be1d-a9ee173a5dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:55.151286 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:55.166084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0fb2c606-3a4a-4e17-be1d-a9ee173a5dba None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:23:55.167490 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0fb2c606-3a4a-4e17-be1d-a9ee173a5dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:23:55.246313 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 624/1248] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:23:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:23:55.273163 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168bb70-408a-487a-a318-c0a89a6fc787 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:55.294542 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:55.393062 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5168bb70-408a-487a-a318-c0a89a6fc787 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:55.435359 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185f01e-5641-4edf-9d02-6734fe3ab400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:55.436280 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185f01e-5641-4edf-9d02-6734fe3ab400 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:55.436601 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185f01e-5641-4edf-9d02-6734fe3ab400 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:55.436898 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2185f01e-5641-4edf-9d02-6734fe3ab400 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:55.448438 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2185f01e-5641-4edf-9d02-6734fe3ab400 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:23:55.457072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2185f01e-5641-4edf-9d02-6734fe3ab400 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:23:55.457072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:23:55.457072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:23:55.457072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:23:55.457072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:23:55.457072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:23:55.457072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:23:55.457072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:23:55.457072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:23:55.457072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:23:55.457072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:23:55.457072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:23:55.457072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:23:55.457072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:23:55.457072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:23:55.457072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:23:55.458946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:23:55.458946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:23:55.458946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:23:55.458946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:23:55.458946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:23:55.458946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:23:55.458946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:23:55.458946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:23:55.458946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:23:55.458946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:23:55.458946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:23:55.458946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:23:55.458946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:23:55.458946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:23:55.458946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:23:55.458946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:23:55.460045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:23:55.460045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:23:55.460045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:23:55.460045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:23:55.460045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:23:55.460045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:23:55.460045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:23:55.460045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:23:55.460045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:23:55.460045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:23:55.460045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:23:55.460045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:23:55.460045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 625/1249] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:23:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 29 12:23:55.464704 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2c606-3a4a-4e17-be1d-a9ee173a5dba None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:55.575340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb2c606-3a4a-4e17-be1d-a9ee173a5dba None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:55.614058 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:55.620496 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0539a832-e319-4279-b0da-a23e374472f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:55.621407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539a832-e319-4279-b0da-a23e374472f8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:55.621699 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539a832-e319-4279-b0da-a23e374472f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:55.621982 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0539a832-e319-4279-b0da-a23e374472f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:55.622847 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0539a832-e319-4279-b0da-a23e374472f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:23:55.623164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0539a832-e319-4279-b0da-a23e374472f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:55.624681 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0539a832-e319-4279-b0da-a23e374472f8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:55.634101 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 625/1250] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:23:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:23:55.639340 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185f01e-5641-4edf-9d02-6734fe3ab400 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:55.728869 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:55.754662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2185f01e-5641-4edf-9d02-6734fe3ab400 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:55.793899 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c97e16-7d9c-4e15-9aaa-1aeb6120b411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:55.794857 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c97e16-7d9c-4e15-9aaa-1aeb6120b411 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:55.795141 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c97e16-7d9c-4e15-9aaa-1aeb6120b411 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:55.795538 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c97e16-7d9c-4e15-9aaa-1aeb6120b411 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:55.807754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c97e16-7d9c-4e15-9aaa-1aeb6120b411 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:55.809210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c97e16-7d9c-4e15-9aaa-1aeb6120b411 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:55.827961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-16c97e16-7d9c-4e15-9aaa-1aeb6120b411 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:23:55.827961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:23:55.827961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:23:55.827961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:23:55.827961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:23:55.827961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:23:55.827961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:23:55.827961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:23:55.827961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:23:55.827961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:23:55.827961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:23:55.827961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:23:55.827961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:23:55.827961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:23:55.827961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:23:55.829083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:23:55.829083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:23:55.829083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:23:55.829083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:23:55.829083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:23:55.829083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:23:55.829083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:23:55.829083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:23:55.829083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:23:55.829083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:23:55.829083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:23:55.829083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:23:55.829083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:23:55.829083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:23:55.829083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:23:55.829083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:23:55.829083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:23:55.830208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:23:55.830208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:23:55.830208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:23:55.830208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:23:55.830208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 626/1251] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:23:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:23:55.836382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0539a832-e319-4279-b0da-a23e374472f8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:55.947380 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0539a832-e319-4279-b0da-a23e374472f8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:55.990630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b066dcc-7b9f-4e27-a78a-35437f468a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:55.991224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b066dcc-7b9f-4e27-a78a-35437f468a75 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:55.991506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b066dcc-7b9f-4e27-a78a-35437f468a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:55.991779 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b066dcc-7b9f-4e27-a78a-35437f468a75 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:55.992496 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b066dcc-7b9f-4e27-a78a-35437f468a75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:23:55.992808 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b066dcc-7b9f-4e27-a78a-35437f468a75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:55.994176 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b066dcc-7b9f-4e27-a78a-35437f468a75 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:56.010170 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 626/1252] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:23:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:23:56.015981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c97e16-7d9c-4e15-9aaa-1aeb6120b411 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:56.138416 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c97e16-7d9c-4e15-9aaa-1aeb6120b411 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:56.169012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:56.178615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b791d-4e9c-4070-bcdb-f17acda2d262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:56.179359 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b791d-4e9c-4070-bcdb-f17acda2d262 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:56.179650 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b791d-4e9c-4070-bcdb-f17acda2d262 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:56.180000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b791d-4e9c-4070-bcdb-f17acda2d262 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:56.189275 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b791d-4e9c-4070-bcdb-f17acda2d262 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:56.191093 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b791d-4e9c-4070-bcdb-f17acda2d262 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:56.210160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-987b791d-4e9c-4070-bcdb-f17acda2d262 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 12:23:56.210160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:23:56.210160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:23:56.210160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:23:56.210160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:23:56.210160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:23:56.210160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:23:56.210160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:23:56.210160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:23:56.210160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:23:56.210160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:23:56.210160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:23:56.210160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:23:56.210160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:23:56.210160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:23:56.210160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:23:56.211413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:23:56.211413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:23:56.211413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:23:56.211413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:23:56.211413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:23:56.211413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:23:56.211413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:23:56.211413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:23:56.211413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:23:56.211413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:23:56.211413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:23:56.211413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:23:56.211413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:23:56.211413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:23:56.211413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:23:56.211413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:23:56.211413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:23:56.211413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:23:56.212718 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:23:56.212718 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:23:56.212718 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:23:56.212718 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:23:56.212718 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:23:56.212718 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:23:56.212718 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:23:56.212718 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:23:56.212718 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 12:23:56.212718 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:23:56.212718 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 627/1253] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:23:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:23:56.219684 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b066dcc-7b9f-4e27-a78a-35437f468a75 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:56.278386 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:56.327608 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b066dcc-7b9f-4e27-a78a-35437f468a75 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:56.373889 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dfa310-4ab0-4790-a910-1037d43d0634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:56.374705 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfa310-4ab0-4790-a910-1037d43d0634 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:56.374996 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfa310-4ab0-4790-a910-1037d43d0634 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:56.375283 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dfa310-4ab0-4790-a910-1037d43d0634 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:56.376336 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dfa310-4ab0-4790-a910-1037d43d0634 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:23:56.376752 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dfa310-4ab0-4790-a910-1037d43d0634 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:56.378522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dfa310-4ab0-4790-a910-1037d43d0634 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:56.399338 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 627/1254] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Jun 29 12:23:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:23:56.406524 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b791d-4e9c-4070-bcdb-f17acda2d262 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:56.523431 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b791d-4e9c-4070-bcdb-f17acda2d262 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:56.557690 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b8a0a-a389-4444-97be-c27db7d7dd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:56.558370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b8a0a-a389-4444-97be-c27db7d7dd2e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:56.558607 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b8a0a-a389-4444-97be-c27db7d7dd2e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:56.558887 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b8a0a-a389-4444-97be-c27db7d7dd2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:56.627686 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:56.645337 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b8a0a-a389-4444-97be-c27db7d7dd2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:56.647580 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b8a0a-a389-4444-97be-c27db7d7dd2e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:56.764543 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 628/1255] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:23:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 12:23:56.771716 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dfa310-4ab0-4790-a910-1037d43d0634 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:56.882010 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dfa310-4ab0-4790-a910-1037d43d0634 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:56.899068 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:56.923795 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79e973-f0cf-46be-bdcc-e9a14bb2eada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:56.924592 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79e973-f0cf-46be-bdcc-e9a14bb2eada None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:56.924875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79e973-f0cf-46be-bdcc-e9a14bb2eada None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:56.925152 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79e973-f0cf-46be-bdcc-e9a14bb2eada None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:56.936580 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc79e973-f0cf-46be-bdcc-e9a14bb2eada None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:56.938163 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc79e973-f0cf-46be-bdcc-e9a14bb2eada None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:56.957749 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-cc79e973-f0cf-46be-bdcc-e9a14bb2eada None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:23:56.957749 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:23:56.957749 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:23:56.957749 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:23:56.957749 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:23:56.957749 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:23:56.957749 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:23:56.957749 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:23:56.957749 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:23:56.957749 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:23:56.957749 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:23:56.957749 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:23:56.957749 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:23:56.957749 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:23:56.957749 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:23:56.959415 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:23:56.959415 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:23:56.959415 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:23:56.959415 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:23:56.959415 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:23:56.959415 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:23:56.959415 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:23:56.959415 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:23:56.959415 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:23:56.959415 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:23:56.959415 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:23:56.959415 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:23:56.959415 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:23:56.959415 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:23:56.959415 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:23:56.959415 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:23:56.959415 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:23:56.960338 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:23:56.960338 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:23:56.960338 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:23:56.960338 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:23:56.960338 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 628/1256] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:23:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:23:56.966214 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b8a0a-a389-4444-97be-c27db7d7dd2e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:57.064953 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:57.078855 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b8a0a-a389-4444-97be-c27db7d7dd2e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:57.122726 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b691d3-65b4-497d-bdbf-070684a17a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:57.123477 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b691d3-65b4-497d-bdbf-070684a17a58 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:57.123802 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b691d3-65b4-497d-bdbf-070684a17a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:57.124058 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b691d3-65b4-497d-bdbf-070684a17a58 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:57.124890 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b691d3-65b4-497d-bdbf-070684a17a58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:23:57.125196 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b691d3-65b4-497d-bdbf-070684a17a58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:57.126662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b691d3-65b4-497d-bdbf-070684a17a58 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:57.141013 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 629/1257] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:23:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:23:57.147627 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79e973-f0cf-46be-bdcc-e9a14bb2eada None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:57.264301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79e973-f0cf-46be-bdcc-e9a14bb2eada None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:57.304639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518e554-dbbe-4e6c-ade5-b8636b83cd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:57.305506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518e554-dbbe-4e6c-ade5-b8636b83cd0c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:57.305788 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518e554-dbbe-4e6c-ade5-b8636b83cd0c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:57.306164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518e554-dbbe-4e6c-ade5-b8636b83cd0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:57.313351 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1518e554-dbbe-4e6c-ade5-b8636b83cd0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:57.314916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1518e554-dbbe-4e6c-ade5-b8636b83cd0c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:57.330474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-1518e554-dbbe-4e6c-ade5-b8636b83cd0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:23:57.330474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:23:57.330474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:23:57.330474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:23:57.330474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:23:57.330474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:23:57.330474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:23:57.330474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:23:57.330474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:23:57.330474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:23:57.330474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:23:57.330474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:23:57.330474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:23:57.330474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:23:57.330474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:23:57.331572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:23:57.331572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:23:57.331572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:23:57.331572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:23:57.331572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:23:57.331572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:23:57.331572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:23:57.331572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:23:57.331572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:23:57.331572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:23:57.331572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:23:57.331572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:23:57.331572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:23:57.331572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:23:57.331572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:23:57.331572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:23:57.331572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:23:57.332548 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:23:57.332548 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:23:57.332548 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:23:57.332548 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:23:57.332548 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:23:57.332548 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:23:57.332548 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:23:57.332548 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:23:57.332548 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 629/1258] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:23:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:23:57.339355 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b691d3-65b4-497d-bdbf-070684a17a58 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:57.455659 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b691d3-65b4-497d-bdbf-070684a17a58 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:57.492828 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0152582-8fd7-4b37-9d1d-8540d473dafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:57.493507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0152582-8fd7-4b37-9d1d-8540d473dafe None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:57.493792 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0152582-8fd7-4b37-9d1d-8540d473dafe None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:57.494069 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0152582-8fd7-4b37-9d1d-8540d473dafe None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:57.494909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0152582-8fd7-4b37-9d1d-8540d473dafe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:23:57.495219 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0152582-8fd7-4b37-9d1d-8540d473dafe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:57.496669 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0152582-8fd7-4b37-9d1d-8540d473dafe None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:57.510476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 630/1259] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:23:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:23:57.516831 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518e554-dbbe-4e6c-ade5-b8636b83cd0c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:57.544740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:57.623799 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518e554-dbbe-4e6c-ade5-b8636b83cd0c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:57.661020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c44eee-e8d2-43e8-8499-a858b3d3affc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:57.662097 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c44eee-e8d2-43e8-8499-a858b3d3affc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:57.662415 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c44eee-e8d2-43e8-8499-a858b3d3affc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:57.662725 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c44eee-e8d2-43e8-8499-a858b3d3affc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:57.663568 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c44eee-e8d2-43e8-8499-a858b3d3affc None admin] RBAC: Adding query filter params (role.id=3a7f4bf556344407a6d9540c89d13641, scope.project.id=f043ade9e4f748d4adc010ffb0a47a46, user.id=c98135cacb9d4f22846ad33500b9bd1f) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:23:57.663881 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c44eee-e8d2-43e8-8499-a858b3d3affc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:23:57.665329 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c44eee-e8d2-43e8-8499-a858b3d3affc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:23:57.682523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 630/1260] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:23:57 2022] GET /identity/v3/role_assignments?user.id=c98135cacb9d4f22846ad33500b9bd1f&scope.project.id=f043ade9e4f748d4adc010ffb0a47a46&role.id=3a7f4bf556344407a6d9540c89d13641 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:23:59.746779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f8d77-8e5a-4b6c-867a-fc822acd2c64 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:59.747293 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f8d77-8e5a-4b6c-867a-fc822acd2c64 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:59.747293 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f8d77-8e5a-4b6c-867a-fc822acd2c64 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:59.748621 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 631/1261] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:23:59.753892 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af13580-84d7-46a2-9a00-6470fad12cba None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:59.754301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af13580-84d7-46a2-9a00-6470fad12cba None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:59.754491 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af13580-84d7-46a2-9a00-6470fad12cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:23:59.771457 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1af13580-84d7-46a2-9a00-6470fad12cba None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:23:59.772933 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1af13580-84d7-46a2-9a00-6470fad12cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:23:59.837123 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 631/1262] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:23:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:23:59.859558 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f8d77-8e5a-4b6c-867a-fc822acd2c64 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:23:59.870768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:23:59.960221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f8d77-8e5a-4b6c-867a-fc822acd2c64 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:23:59.993119 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00cf441-032a-4ee1-82fe-34d1d84cef0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:23:59.993890 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00cf441-032a-4ee1-82fe-34d1d84cef0a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:23:59.994171 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00cf441-032a-4ee1-82fe-34d1d84cef0a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:23:59.994473 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00cf441-032a-4ee1-82fe-34d1d84cef0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:00.004316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00cf441-032a-4ee1-82fe-34d1d84cef0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:24:00.012187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b00cf441-032a-4ee1-82fe-34d1d84cef0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:24:00.012187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:24:00.012187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:24:00.012187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:24:00.012187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:24:00.012187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:24:00.012187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:24:00.012187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:24:00.012187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:24:00.012187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:24:00.012187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:24:00.012187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:24:00.012187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:24:00.012187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:24:00.012187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:24:00.012187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:24:00.013312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:24:00.013312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:24:00.013312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:24:00.013312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:24:00.013312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:24:00.013312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:24:00.013312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:24:00.013312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:24:00.013312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:24:00.013312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:24:00.013312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:24:00.013312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:24:00.013312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:24:00.013312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:24:00.013312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:24:00.013312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:24:00.014565 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:24:00.014565 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:24:00.014565 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:24:00.014565 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:24:00.014565 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:24:00.014565 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:24:00.014565 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:24:00.014565 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:24:00.014565 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:24:00.014565 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:24:00.014565 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:24:00.014565 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:24:00.014565 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 632/1263] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:23:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:24:00.019401 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af13580-84d7-46a2-9a00-6470fad12cba None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:00.037791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:24:00.124333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af13580-84d7-46a2-9a00-6470fad12cba None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:00.159935 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef11409-5907-48aa-ac4a-c742188f7e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:00.160593 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef11409-5907-48aa-ac4a-c742188f7e3c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:00.160876 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef11409-5907-48aa-ac4a-c742188f7e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:00.161646 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef11409-5907-48aa-ac4a-c742188f7e3c None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:00.162464 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef11409-5907-48aa-ac4a-c742188f7e3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:24:00.162776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef11409-5907-48aa-ac4a-c742188f7e3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:00.164138 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef11409-5907-48aa-ac4a-c742188f7e3c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:00.172152 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 632/1264] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:24:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:24:00.178783 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00cf441-032a-4ee1-82fe-34d1d84cef0a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:00.246313 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:24:00.280325 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00cf441-032a-4ee1-82fe-34d1d84cef0a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:00.312848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab57954-e588-4a6f-8450-e3c521857907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:00.313628 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab57954-e588-4a6f-8450-e3c521857907 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:00.313909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab57954-e588-4a6f-8450-e3c521857907 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:00.314184 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab57954-e588-4a6f-8450-e3c521857907 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:00.324506 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab57954-e588-4a6f-8450-e3c521857907 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:00.325932 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab57954-e588-4a6f-8450-e3c521857907 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:00.342909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-aab57954-e588-4a6f-8450-e3c521857907 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:24:00.342909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:24:00.342909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:24:00.342909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:24:00.342909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:24:00.342909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:24:00.342909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:24:00.342909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:24:00.342909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:24:00.342909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:24:00.342909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:24:00.342909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:24:00.342909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:24:00.342909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:24:00.342909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:24:00.344002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:24:00.344002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:24:00.344002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:24:00.344002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:24:00.344002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:24:00.344002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:24:00.344002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:24:00.344002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:24:00.344002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:24:00.344002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:24:00.344002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:24:00.344002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:24:00.344002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:24:00.344002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:24:00.344002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:24:00.344002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:24:00.344002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:24:00.344914 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:24:00.344914 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:24:00.344914 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:24:00.344914 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:24:00.344914 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 633/1265] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:24:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:24:00.350567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef11409-5907-48aa-ac4a-c742188f7e3c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:00.429893 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:24:00.448501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef11409-5907-48aa-ac4a-c742188f7e3c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:00.481057 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800482c-22a9-41f4-9c71-0c99e02c36eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:00.481705 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800482c-22a9-41f4-9c71-0c99e02c36eb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:00.481990 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800482c-22a9-41f4-9c71-0c99e02c36eb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:00.482266 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800482c-22a9-41f4-9c71-0c99e02c36eb None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:00.483014 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800482c-22a9-41f4-9c71-0c99e02c36eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:24:00.483320 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800482c-22a9-41f4-9c71-0c99e02c36eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:00.484694 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800482c-22a9-41f4-9c71-0c99e02c36eb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:00.496120 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 633/1266] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:24:00.501812 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab57954-e588-4a6f-8450-e3c521857907 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:00.598321 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab57954-e588-4a6f-8450-e3c521857907 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:00.617805 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:24:00.632549 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72236cd-f0a1-4519-bde8-da7bb204ea09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:00.633331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72236cd-f0a1-4519-bde8-da7bb204ea09 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:00.633617 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72236cd-f0a1-4519-bde8-da7bb204ea09 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:00.633893 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72236cd-f0a1-4519-bde8-da7bb204ea09 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:00.641113 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72236cd-f0a1-4519-bde8-da7bb204ea09 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:00.642578 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72236cd-f0a1-4519-bde8-da7bb204ea09 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:00.657331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f72236cd-f0a1-4519-bde8-da7bb204ea09 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 12:24:00.657331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:24:00.657331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:24:00.657331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:24:00.657331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:24:00.657331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:24:00.657331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:24:00.657331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:24:00.657331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:24:00.657331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:24:00.657331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:24:00.657331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:24:00.657331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:24:00.657331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:24:00.657331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:24:00.657331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:24:00.658475 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:24:00.658475 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:24:00.658475 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:24:00.658475 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:24:00.658475 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:24:00.658475 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:24:00.658475 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:24:00.658475 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:24:00.658475 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:24:00.658475 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:24:00.658475 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:24:00.658475 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:24:00.658475 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:24:00.658475 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:24:00.658475 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:24:00.658475 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:24:00.658475 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:24:00.658475 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:24:00.659815 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:24:00.659815 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:24:00.659815 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:24:00.659815 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:24:00.659815 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:24:00.659815 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:24:00.659815 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:24:00.659815 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:24:00.659815 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 12:24:00.659815 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:24:00.659815 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 634/1267] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:24:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:24:00.664864 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800482c-22a9-41f4-9c71-0c99e02c36eb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:00.760344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800482c-22a9-41f4-9c71-0c99e02c36eb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:00.795483 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d44952-6790-464e-af96-73831edf0c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:00.796133 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d44952-6790-464e-af96-73831edf0c76 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:00.796418 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d44952-6790-464e-af96-73831edf0c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:00.796697 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d44952-6790-464e-af96-73831edf0c76 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:00.797524 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d44952-6790-464e-af96-73831edf0c76 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:24:00.797834 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d44952-6790-464e-af96-73831edf0c76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:00.799394 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d44952-6790-464e-af96-73831edf0c76 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:00.808397 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:24:00.816669 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 634/1268] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Jun 29 12:24:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:24:00.822373 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72236cd-f0a1-4519-bde8-da7bb204ea09 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:00.920171 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72236cd-f0a1-4519-bde8-da7bb204ea09 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:00.951950 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4a563-712e-4f68-a0a6-a8fa930067c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:00.952720 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4a563-712e-4f68-a0a6-a8fa930067c2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:00.953004 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4a563-712e-4f68-a0a6-a8fa930067c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:00.953279 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4a563-712e-4f68-a0a6-a8fa930067c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:00.963728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4a563-712e-4f68-a0a6-a8fa930067c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:00.965155 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4a563-712e-4f68-a0a6-a8fa930067c2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:00.982385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9ca4a563-712e-4f68-a0a6-a8fa930067c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:24:00.982385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:24:00.982385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:24:00.982385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:24:00.982385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:24:00.982385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:24:00.982385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:24:00.982385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:24:00.982385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:24:00.982385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:24:00.982385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:24:00.982385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:24:00.982385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:24:00.982385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:24:00.982385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:24:00.983316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:24:00.983316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:24:00.983316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:24:00.983316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:24:00.983316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:24:00.983316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:24:00.983316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:24:00.983316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:24:00.983316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:24:00.983316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:24:00.983316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:24:00.983316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:24:00.983316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:24:00.983316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:24:00.983316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:24:00.983316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:24:00.983316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:24:00.984271 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:24:00.984271 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:24:00.984271 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:24:00.984271 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:24:00.984271 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 635/1269] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:24:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:24:00.989797 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d44952-6790-464e-af96-73831edf0c76 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:01.087320 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d44952-6790-464e-af96-73831edf0c76 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:01.121541 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28255e4a-a31f-46d0-935c-615f5b4006ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:01.122189 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28255e4a-a31f-46d0-935c-615f5b4006ee None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:01.122499 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28255e4a-a31f-46d0-935c-615f5b4006ee None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:01.122777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28255e4a-a31f-46d0-935c-615f5b4006ee None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:01.123485 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28255e4a-a31f-46d0-935c-615f5b4006ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:24:01.123790 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28255e4a-a31f-46d0-935c-615f5b4006ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:01.125148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28255e4a-a31f-46d0-935c-615f5b4006ee None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:01.135980 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 635/1270] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:24:01.141692 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4a563-712e-4f68-a0a6-a8fa930067c2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:01.173250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:24:01.238904 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4a563-712e-4f68-a0a6-a8fa930067c2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:01.271141 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc09aa-befb-40d2-9075-b9d206b72af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:01.271991 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc09aa-befb-40d2-9075-b9d206b72af7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:01.272530 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc09aa-befb-40d2-9075-b9d206b72af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:01.272806 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc09aa-befb-40d2-9075-b9d206b72af7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:01.279231 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bc09aa-befb-40d2-9075-b9d206b72af7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:01.280661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bc09aa-befb-40d2-9075-b9d206b72af7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:01.294848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-12bc09aa-befb-40d2-9075-b9d206b72af7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:24:01.294848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:24:01.294848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:24:01.294848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:24:01.294848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:24:01.294848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:24:01.294848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:24:01.294848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:24:01.294848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:24:01.294848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:24:01.294848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:24:01.294848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:24:01.294848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:24:01.294848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:24:01.294848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:24:01.295874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:24:01.295874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:24:01.295874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:24:01.295874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:24:01.295874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:24:01.295874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:24:01.295874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:24:01.295874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:24:01.295874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:24:01.295874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:24:01.295874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:24:01.295874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:24:01.295874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:24:01.295874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:24:01.295874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:24:01.295874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:24:01.295874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:24:01.297025 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:24:01.297025 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:24:01.297025 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:24:01.297025 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:24:01.297025 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:24:01.297025 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:24:01.297025 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:24:01.297025 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:24:01.297025 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 636/1271] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:24:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:24:01.301690 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28255e4a-a31f-46d0-935c-615f5b4006ee None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:01.358002 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:24:01.399762 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28255e4a-a31f-46d0-935c-615f5b4006ee None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:01.431469 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4048aaef-d05f-4465-9ea6-4452e3017b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:01.432052 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048aaef-d05f-4465-9ea6-4452e3017b3a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:01.432335 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048aaef-d05f-4465-9ea6-4452e3017b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:01.432611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4048aaef-d05f-4465-9ea6-4452e3017b3a None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:01.433382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4048aaef-d05f-4465-9ea6-4452e3017b3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:24:01.433690 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4048aaef-d05f-4465-9ea6-4452e3017b3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:01.435141 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4048aaef-d05f-4465-9ea6-4452e3017b3a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:01.447288 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 636/1272] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:24:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:24:01.453168 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc09aa-befb-40d2-9075-b9d206b72af7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:01.541143 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:24:01.550008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc09aa-befb-40d2-9075-b9d206b72af7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:01.582862 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e884a49b-6a55-478c-9748-c1acb5409cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:01.584455 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884a49b-6a55-478c-9748-c1acb5409cde None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:01.584741 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884a49b-6a55-478c-9748-c1acb5409cde None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:01.585017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884a49b-6a55-478c-9748-c1acb5409cde None admin] PATH_INFO: `/v3/projects/f043ade9e4f748d4adc010ffb0a47a46/users/c98135cacb9d4f22846ad33500b9bd1f/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:01.587006 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e884a49b-6a55-478c-9748-c1acb5409cde None admin] RBAC: Authorizing `identity:create_grant(project_id=f043ade9e4f748d4adc010ffb0a47a46, user_id=c98135cacb9d4f22846ad33500b9bd1f, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:01.588427 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e884a49b-6a55-478c-9748-c1acb5409cde None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:01.600319 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 637/1273] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jun 29 12:24:01 2022] PUT /identity/v3/projects/f043ade9e4f748d4adc010ffb0a47a46/users/c98135cacb9d4f22846ad33500b9bd1f/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:24:03.619444 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef02b0-f0f6-45d3-9b82-ffdd73fe7808 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:03.619988 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef02b0-f0f6-45d3-9b82-ffdd73fe7808 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:03.619988 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef02b0-f0f6-45d3-9b82-ffdd73fe7808 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:03.621276 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 637/1274] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:24:03.626266 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9836230-4433-4b17-9aef-00bf13867962 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:03.626692 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9836230-4433-4b17-9aef-00bf13867962 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:03.626874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9836230-4433-4b17-9aef-00bf13867962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:03.643851 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c9836230-4433-4b17-9aef-00bf13867962 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:24:03.645038 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c9836230-4433-4b17-9aef-00bf13867962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:24:03.709975 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 638/1275] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:24:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:24:03.733707 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef02b0-f0f6-45d3-9b82-ffdd73fe7808 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:03.830414 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef02b0-f0f6-45d3-9b82-ffdd73fe7808 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:03.846901 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:24:03.863669 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3706d3-b629-4d5f-ab0d-e2d63076e706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:03.864510 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3706d3-b629-4d5f-ab0d-e2d63076e706 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:03.864790 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3706d3-b629-4d5f-ab0d-e2d63076e706 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:03.865065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3706d3-b629-4d5f-ab0d-e2d63076e706 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:03.875097 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3706d3-b629-4d5f-ab0d-e2d63076e706 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 29 12:24:03.883069 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ad3706d3-b629-4d5f-ab0d-e2d63076e706 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:24:03.883069 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:24:03.883069 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:24:03.883069 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:24:03.883069 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:24:03.883069 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:24:03.883069 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:24:03.883069 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:24:03.883069 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:24:03.883069 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:24:03.883069 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:24:03.883069 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:24:03.883069 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 29 12:24:03.883069 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 29 12:24:03.883069 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 29 12:24:03.883069 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 29 12:24:03.885318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 29 12:24:03.885318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 29 12:24:03.885318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 29 12:24:03.885318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 29 12:24:03.885318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:24:03.885318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:24:03.885318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:24:03.885318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:24:03.885318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:24:03.885318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:24:03.885318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:24:03.885318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:24:03.885318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:24:03.885318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:24:03.885318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:24:03.885318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:24:03.886523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:24:03.886523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:24:03.886523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:24:03.886523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:24:03.886523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 29 12:24:03.886523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 29 12:24:03.886523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 29 12:24:03.886523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 29 12:24:03.886523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 29 12:24:03.886523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 29 12:24:03.886523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 29 12:24:03.886523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:24:03.886523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 638/1276] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jun 29 12:24:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:24:03.890099 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9836230-4433-4b17-9aef-00bf13867962 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:03.970237 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:24:03.989092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9836230-4433-4b17-9aef-00bf13867962 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:04.022611 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e65d7-80b4-4775-93c4-f873c5286d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:04.023273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e65d7-80b4-4775-93c4-f873c5286d27 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:04.023559 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e65d7-80b4-4775-93c4-f873c5286d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:04.023837 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e65d7-80b4-4775-93c4-f873c5286d27 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:04.024584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e65d7-80b4-4775-93c4-f873c5286d27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:24:04.024896 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e65d7-80b4-4775-93c4-f873c5286d27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:04.026256 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e65d7-80b4-4775-93c4-f873c5286d27 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:04.034072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 639/1277] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jun 29 12:24:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:24:04.039919 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3706d3-b629-4d5f-ab0d-e2d63076e706 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:04.143384 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3706d3-b629-4d5f-ab0d-e2d63076e706 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:04.175293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18abd781-9b5f-4af7-a1c3-f436e526e20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:04.176071 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abd781-9b5f-4af7-a1c3-f436e526e20d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:04.176356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abd781-9b5f-4af7-a1c3-f436e526e20d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:04.177150 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18abd781-9b5f-4af7-a1c3-f436e526e20d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:04.187280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18abd781-9b5f-4af7-a1c3-f436e526e20d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:04.188725 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18abd781-9b5f-4af7-a1c3-f436e526e20d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:04.205916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-18abd781-9b5f-4af7-a1c3-f436e526e20d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:24:04.205916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:24:04.205916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:24:04.205916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:24:04.205916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:24:04.205916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:24:04.205916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:24:04.205916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:24:04.205916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:24:04.205916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:24:04.205916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:24:04.205916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:24:04.205916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:24:04.205916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:24:04.205916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:24:04.207050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:24:04.207050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:24:04.207050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:24:04.207050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:24:04.207050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:24:04.207050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:24:04.207050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:24:04.207050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:24:04.207050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:24:04.207050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:24:04.207050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:24:04.207050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:24:04.207050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:24:04.207050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:24:04.207050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:24:04.207050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:24:04.207050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:24:04.208282 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:24:04.208282 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:24:04.208282 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:24:04.208282 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:24:04.210599 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 639/1278] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:24:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:24:04.213938 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e65d7-80b4-4775-93c4-f873c5286d27 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:04.313331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e65d7-80b4-4775-93c4-f873c5286d27 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:04.346331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285b035-cc10-495e-ae5a-49bcb1923f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:04.347020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285b035-cc10-495e-ae5a-49bcb1923f58 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:04.347305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285b035-cc10-495e-ae5a-49bcb1923f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:04.347585 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285b035-cc10-495e-ae5a-49bcb1923f58 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:04.348301 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e285b035-cc10-495e-ae5a-49bcb1923f58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:24:04.349449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e285b035-cc10-495e-ae5a-49bcb1923f58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:04.350859 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e285b035-cc10-495e-ae5a-49bcb1923f58 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:04.362874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 640/1279] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:24:04.369250 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18abd781-9b5f-4af7-a1c3-f436e526e20d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:04.399660 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:24:04.468908 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18abd781-9b5f-4af7-a1c3-f436e526e20d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:04.500995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1eb5a-09f6-478d-8904-d4aa3a9cc513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:04.501779 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1eb5a-09f6-478d-8904-d4aa3a9cc513 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:04.502063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1eb5a-09f6-478d-8904-d4aa3a9cc513 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:04.502339 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a1eb5a-09f6-478d-8904-d4aa3a9cc513 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:04.510076 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a1eb5a-09f6-478d-8904-d4aa3a9cc513 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:04.511537 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a1eb5a-09f6-478d-8904-d4aa3a9cc513 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:04.526356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-97a1eb5a-09f6-478d-8904-d4aa3a9cc513 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 29 12:24:04.526356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:24:04.526356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:24:04.526356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:24:04.526356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:24:04.526356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:24:04.526356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:24:04.526356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:24:04.526356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:24:04.526356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:24:04.526356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:24:04.526356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:24:04.526356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 29 12:24:04.526356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 29 12:24:04.526356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 29 12:24:04.526356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 29 12:24:04.527549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:24:04.527549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:24:04.527549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 29 12:24:04.527549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:24:04.527549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 29 12:24:04.527549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 29 12:24:04.527549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:24:04.527549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:24:04.527549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:24:04.527549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:24:04.527549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:24:04.527549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:24:04.527549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:24:04.527549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:24:04.527549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:24:04.527549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:24:04.527549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:24:04.527549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:24:04.528931 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:24:04.528931 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:24:04.528931 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 29 12:24:04.528931 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 29 12:24:04.528931 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 29 12:24:04.528931 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 29 12:24:04.528931 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 29 12:24:04.528931 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 29 12:24:04.528931 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 29 12:24:04.528931 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:24:04.528931 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 640/1280] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:24:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:24:04.533364 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285b035-cc10-495e-ae5a-49bcb1923f58 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:04.537187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:24:04.628908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285b035-cc10-495e-ae5a-49bcb1923f58 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:04.662575 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b2e7c-f5d7-4445-bbd7-0102c7625567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:04.663178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b2e7c-f5d7-4445-bbd7-0102c7625567 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:04.663466 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b2e7c-f5d7-4445-bbd7-0102c7625567 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:04.663745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4b2e7c-f5d7-4445-bbd7-0102c7625567 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:04.664572 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b2e7c-f5d7-4445-bbd7-0102c7625567 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:24:04.664885 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b2e7c-f5d7-4445-bbd7-0102c7625567 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:04.666342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4b2e7c-f5d7-4445-bbd7-0102c7625567 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:04.682764 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 641/1281] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Jun 29 12:24:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:24:04.688395 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1eb5a-09f6-478d-8904-d4aa3a9cc513 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:04.775503 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:24:04.784522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a1eb5a-09f6-478d-8904-d4aa3a9cc513 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:04.817762 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2da0a2-52a3-48b8-8f90-30a264274bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:04.818391 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2da0a2-52a3-48b8-8f90-30a264274bbe None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:04.818704 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2da0a2-52a3-48b8-8f90-30a264274bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:04.818982 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2da0a2-52a3-48b8-8f90-30a264274bbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:04.882344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2da0a2-52a3-48b8-8f90-30a264274bbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:04.883803 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2da0a2-52a3-48b8-8f90-30a264274bbe None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:04.972817 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:24:04.983832 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 641/1282] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jun 29 12:24:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 29 12:24:04.989678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b2e7c-f5d7-4445-bbd7-0102c7625567 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:05.048891 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:24:05.087185 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4b2e7c-f5d7-4445-bbd7-0102c7625567 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:05.124280 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372900f9-485e-4dc3-9eb2-179bb631a36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:05.125062 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372900f9-485e-4dc3-9eb2-179bb631a36a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:05.125345 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372900f9-485e-4dc3-9eb2-179bb631a36a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:05.125621 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372900f9-485e-4dc3-9eb2-179bb631a36a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:05.135779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372900f9-485e-4dc3-9eb2-179bb631a36a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:05.137204 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372900f9-485e-4dc3-9eb2-179bb631a36a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:05.154684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-372900f9-485e-4dc3-9eb2-179bb631a36a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:24:05.154684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:24:05.154684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:24:05.154684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:24:05.154684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:24:05.154684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:24:05.154684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:24:05.154684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:24:05.154684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:24:05.154684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:24:05.154684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:24:05.154684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:24:05.154684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 29 12:24:05.154684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 29 12:24:05.154684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 29 12:24:05.155750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 29 12:24:05.155750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:24:05.155750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:24:05.155750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:24:05.155750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:24:05.155750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:24:05.155750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:24:05.155750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:24:05.155750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:24:05.155750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:24:05.155750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:24:05.155750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:24:05.155750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:24:05.155750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:24:05.155750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:24:05.155750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:24:05.155750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:24:05.156865 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 29 12:24:05.156865 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 29 12:24:05.156865 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 29 12:24:05.156865 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:24:05.156865 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 642/1283] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:24:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:24:05.161636 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2da0a2-52a3-48b8-8f90-30a264274bbe None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:05.257254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2da0a2-52a3-48b8-8f90-30a264274bbe None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:05.290739 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0798ab-9bca-4cd9-a673-7255163dd71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:05.291382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0798ab-9bca-4cd9-a673-7255163dd71b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:05.291666 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0798ab-9bca-4cd9-a673-7255163dd71b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:05.291941 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0798ab-9bca-4cd9-a673-7255163dd71b None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:05.292648 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0798ab-9bca-4cd9-a673-7255163dd71b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:24:05.292959 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0798ab-9bca-4cd9-a673-7255163dd71b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:05.294314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0798ab-9bca-4cd9-a673-7255163dd71b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:05.305085 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 642/1284] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jun 29 12:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:24:05.310536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372900f9-485e-4dc3-9eb2-179bb631a36a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:05.409189 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372900f9-485e-4dc3-9eb2-179bb631a36a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:05.423162 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:24:05.443823 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537b3aa-0610-40ac-8b44-d2d110cf9c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:05.444591 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537b3aa-0610-40ac-8b44-d2d110cf9c13 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:05.444875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537b3aa-0610-40ac-8b44-d2d110cf9c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:05.445146 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537b3aa-0610-40ac-8b44-d2d110cf9c13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:05.451604 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537b3aa-0610-40ac-8b44-d2d110cf9c13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:05.453027 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537b3aa-0610-40ac-8b44-d2d110cf9c13 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:05.467112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1537b3aa-0610-40ac-8b44-d2d110cf9c13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:24:05.467112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:24:05.467112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:24:05.467112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:24:05.467112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:24:05.467112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:24:05.467112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:24:05.467112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:24:05.467112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:24:05.467112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:24:05.467112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:24:05.467112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:24:05.467112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 29 12:24:05.467112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 29 12:24:05.467112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 29 12:24:05.468190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 29 12:24:05.468190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:24:05.468190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:24:05.468190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:24:05.468190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:24:05.468190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:24:05.468190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:24:05.468190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:24:05.468190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jun 29 12:24:05.468190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:24:05.468190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:24:05.468190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:24:05.468190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:24:05.468190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:24:05.468190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:24:05.468190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:24:05.468190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:24:05.469488 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 29 12:24:05.469488 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 29 12:24:05.469488 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 29 12:24:05.469488 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 29 12:24:05.469488 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 29 12:24:05.469488 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 29 12:24:05.469488 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 29 12:24:05.469488 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jun 29 12:24:05.469488 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 643/1285] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jun 29 12:24:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:24:05.474230 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0798ab-9bca-4cd9-a673-7255163dd71b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:05.524414 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:24:05.573941 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0798ab-9bca-4cd9-a673-7255163dd71b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:05.605425 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba3bbd6-ddc0-47c2-ad77-e35c33cbaa06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:05.606069 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3bbd6-ddc0-47c2-ad77-e35c33cbaa06 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:05.606354 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3bbd6-ddc0-47c2-ad77-e35c33cbaa06 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:05.606681 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba3bbd6-ddc0-47c2-ad77-e35c33cbaa06 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:05.607466 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba3bbd6-ddc0-47c2-ad77-e35c33cbaa06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:24:05.607777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba3bbd6-ddc0-47c2-ad77-e35c33cbaa06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:05.609226 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba3bbd6-ddc0-47c2-ad77-e35c33cbaa06 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:05.621451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 643/1286] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jun 29 12:24:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:24:05.627463 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537b3aa-0610-40ac-8b44-d2d110cf9c13 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:05.725065 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537b3aa-0610-40ac-8b44-d2d110cf9c13 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:05.757387 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c565f136-7c79-4a37-81c1-1df1d0bc3d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:05.758020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565f136-7c79-4a37-81c1-1df1d0bc3d09 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:05.758326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565f136-7c79-4a37-81c1-1df1d0bc3d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:05.758608 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565f136-7c79-4a37-81c1-1df1d0bc3d09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:05.759427 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c565f136-7c79-4a37-81c1-1df1d0bc3d09 None admin] RBAC: Adding query filter params (role.id=3a7f4bf556344407a6d9540c89d13641, scope.project.id=f043ade9e4f748d4adc010ffb0a47a46, user.id=c98135cacb9d4f22846ad33500b9bd1f) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:24:05.759736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c565f136-7c79-4a37-81c1-1df1d0bc3d09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:05.761143 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c565f136-7c79-4a37-81c1-1df1d0bc3d09 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:05.766754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 644/1287] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jun 29 12:24:05 2022] GET /identity/v3/role_assignments?user.id=c98135cacb9d4f22846ad33500b9bd1f&scope.project.id=f043ade9e4f748d4adc010ffb0a47a46&role.id=3a7f4bf556344407a6d9540c89d13641 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:24:07.781950 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde65157-0796-4358-8912-882ea252e6ed None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:07.782487 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde65157-0796-4358-8912-882ea252e6ed None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:07.782584 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde65157-0796-4358-8912-882ea252e6ed None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:07.783899 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 644/1288] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:24:07.789552 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bbf62-0d9e-4895-9b92-06f5475cd3d2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:07.789969 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bbf62-0d9e-4895-9b92-06f5475cd3d2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:07.790106 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564bbf62-0d9e-4895-9b92-06f5475cd3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:07.807092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-564bbf62-0d9e-4895-9b92-06f5475cd3d2 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:24:07.808257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-564bbf62-0d9e-4895-9b92-06f5475cd3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:24:07.837158 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:24:07.873262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 645/1289] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:24:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:24:07.895138 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde65157-0796-4358-8912-882ea252e6ed None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:07.997378 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde65157-0796-4358-8912-882ea252e6ed None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:08.008245 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:24:08.031140 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c4d4c-7d56-402f-85c3-de88dca92bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:08.031926 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c4d4c-7d56-402f-85c3-de88dca92bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:08.032259 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c4d4c-7d56-402f-85c3-de88dca92bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:08.032494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c4d4c-7d56-402f-85c3-de88dca92bd4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:08.033139 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0c4d4c-7d56-402f-85c3-de88dca92bd4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:08.034529 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0c4d4c-7d56-402f-85c3-de88dca92bd4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:08.046177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ca0c4d4c-7d56-402f-85c3-de88dca92bd4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 12:24:08.046177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:24:08.046177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 12:24:08.046177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 12:24:08.046177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 12:24:08.046177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 12:24:08.046177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 12:24:08.046177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:24:08.046177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 12:24:08.046177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:24:08.046177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:24:08.046177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:24:08.046177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:24:08.046177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:24:08.046177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:24:08.046177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:24:08.046177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:24:08.047404 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:24:08.047404 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:24:08.047404 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:24:08.047404 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:24:08.047404 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:24:08.047404 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:24:08.047404 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 12:24:08.047404 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 12:24:08.047404 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 12:24:08.047404 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 12:24:08.047404 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:24:08.047404 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:24:08.047404 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:24:08.047404 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:24:08.047404 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:24:08.047404 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:24:08.048288 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:24:08.048288 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:24:08.048288 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:24:08.048288 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:24:08.048288 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:24:08.048288 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:24:08.048288 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:24:08.048288 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:24:08.048288 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:24:08.048288 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:24:08.048288 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 12:24:08.048288 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:24:08.048288 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 12:24:08.048288 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:24:08.048288 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 645/1290] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jun 29 12:24:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:24:08.054259 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bbf62-0d9e-4895-9b92-06f5475cd3d2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:08.171120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564bbf62-0d9e-4895-9b92-06f5475cd3d2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:08.203780 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e599965-9d4d-4420-afa1-09231fc55318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:08.204439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e599965-9d4d-4420-afa1-09231fc55318 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:08.204724 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e599965-9d4d-4420-afa1-09231fc55318 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:08.204999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e599965-9d4d-4420-afa1-09231fc55318 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:08.205696 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e599965-9d4d-4420-afa1-09231fc55318 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:24:08.206002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e599965-9d4d-4420-afa1-09231fc55318 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:08.207389 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e599965-9d4d-4420-afa1-09231fc55318 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:08.212219 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 646/1291] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Jun 29 12:24:08 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:24:08.217877 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c4d4c-7d56-402f-85c3-de88dca92bd4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:08.315790 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c4d4c-7d56-402f-85c3-de88dca92bd4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:08.347063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02118aa-7c3f-49f4-98b0-81f876d5ce9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:08.347745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02118aa-7c3f-49f4-98b0-81f876d5ce9b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:08.348027 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02118aa-7c3f-49f4-98b0-81f876d5ce9b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:08.348304 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02118aa-7c3f-49f4-98b0-81f876d5ce9b None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:08.349601 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02118aa-7c3f-49f4-98b0-81f876d5ce9b None admin] RBAC: Adding query filter params (type=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:24:08.349903 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02118aa-7c3f-49f4-98b0-81f876d5ce9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:08.351286 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02118aa-7c3f-49f4-98b0-81f876d5ce9b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:08.355735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 646/1292] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Jun 29 12:24:08 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:24:10.342599 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31edda8-651c-4a9e-9c65-e67c656ccc45 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:10.343149 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31edda8-651c-4a9e-9c65-e67c656ccc45 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:10.343149 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31edda8-651c-4a9e-9c65-e67c656ccc45 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:10.344523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 647/1293] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:24:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:24:10.349775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55339701-2fe7-4c68-8e94-7a5fb3aff438 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:10.350176 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55339701-2fe7-4c68-8e94-7a5fb3aff438 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:10.350328 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55339701-2fe7-4c68-8e94-7a5fb3aff438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:10.367095 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-55339701-2fe7-4c68-8e94-7a5fb3aff438 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:24:10.368265 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-55339701-2fe7-4c68-8e94-7a5fb3aff438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:24:10.396981 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:24:10.433550 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 647/1294] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:24:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:24:10.456117 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31edda8-651c-4a9e-9c65-e67c656ccc45 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:10.486724 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:24:10.556471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31edda8-651c-4a9e-9c65-e67c656ccc45 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:10.588839 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2573c1-e13c-469b-ba16-b6deca3055b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:10.589526 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2573c1-e13c-469b-ba16-b6deca3055b2 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:10.589812 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2573c1-e13c-469b-ba16-b6deca3055b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:10.590085 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2573c1-e13c-469b-ba16-b6deca3055b2 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:10.590846 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2573c1-e13c-469b-ba16-b6deca3055b2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:10.592199 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2573c1-e13c-469b-ba16-b6deca3055b2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:10.605969 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 648/1295] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jun 29 12:24:10 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:24:12.605974 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6b855-8bff-4b81-8032-8d709d9271c6 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:12.606567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6b855-8bff-4b81-8032-8d709d9271c6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:12.606567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f6b855-8bff-4b81-8032-8d709d9271c6 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:12.607997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 648/1296] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:24:12.613071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2be11-1aa4-4055-8e99-6b5d96dd875f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:12.613476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2be11-1aa4-4055-8e99-6b5d96dd875f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:12.613590 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2be11-1aa4-4055-8e99-6b5d96dd875f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:12.630601 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a6d2be11-1aa4-4055-8e99-6b5d96dd875f None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:24:12.631791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a6d2be11-1aa4-4055-8e99-6b5d96dd875f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:24:12.702197 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 649/1297] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:24:12 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 29 12:24:12.725448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f6b855-8bff-4b81-8032-8d709d9271c6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:12.829133 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f6b855-8bff-4b81-8032-8d709d9271c6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:12.861532 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe0eb8-33f0-47d2-85ce-2c21d6a562eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:12.862308 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe0eb8-33f0-47d2-85ce-2c21d6a562eb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:12.862625 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe0eb8-33f0-47d2-85ce-2c21d6a562eb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:12.862901 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe0eb8-33f0-47d2-85ce-2c21d6a562eb None admin] PATH_INFO: `/v3/services/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:12.863595 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe0eb8-33f0-47d2-85ce-2c21d6a562eb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:12.864933 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe0eb8-33f0-47d2-85ce-2c21d6a562eb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:12.876573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ccfe0eb8-33f0-47d2-85ce-2c21d6a562eb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 12:24:12.876573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:24:12.876573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 12:24:12.876573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 12:24:12.876573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 12:24:12.876573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 12:24:12.876573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 12:24:12.876573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:24:12.876573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 12:24:12.876573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:24:12.876573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:24:12.876573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:24:12.876573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:24:12.876573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:24:12.876573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:24:12.876573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:24:12.876573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:24:12.877779 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:24:12.877779 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:24:12.877779 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:24:12.877779 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:24:12.877779 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:24:12.877779 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:24:12.877779 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 12:24:12.877779 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 12:24:12.877779 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 12:24:12.877779 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 12:24:12.877779 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:24:12.877779 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:24:12.877779 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:24:12.877779 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:24:12.877779 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:24:12.877779 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:24:12.878841 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:24:12.878841 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:24:12.878841 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:24:12.878841 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:24:12.878841 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:24:12.878841 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:24:12.878841 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:24:12.878841 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:24:12.878841 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:24:12.878841 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:24:12.878841 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 12:24:12.878841 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:24:12.878841 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 12:24:12.878841 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:24:12.878841 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 649/1298] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jun 29 12:24:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:24:12.884180 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2be11-1aa4-4055-8e99-6b5d96dd875f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:12.903079 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:24:12.982746 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d2be11-1aa4-4055-8e99-6b5d96dd875f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:13.015409 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3353f8c-78b9-4240-8685-3edbc1c57919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:13.016061 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3353f8c-78b9-4240-8685-3edbc1c57919 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:13.016342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3353f8c-78b9-4240-8685-3edbc1c57919 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:13.016618 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3353f8c-78b9-4240-8685-3edbc1c57919 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:13.017315 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3353f8c-78b9-4240-8685-3edbc1c57919 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:24:13.017618 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3353f8c-78b9-4240-8685-3edbc1c57919 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:13.019015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3353f8c-78b9-4240-8685-3edbc1c57919 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:13.024870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 650/1299] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Jun 29 12:24:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:24:13.030784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe0eb8-33f0-47d2-85ce-2c21d6a562eb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:13.063277 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:24:13.146242 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe0eb8-33f0-47d2-85ce-2c21d6a562eb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:13.180082 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcd483-f523-4fe3-9188-1d6564a076e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:13.180734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcd483-f523-4fe3-9188-1d6564a076e4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:13.181020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcd483-f523-4fe3-9188-1d6564a076e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:13.181299 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcd483-f523-4fe3-9188-1d6564a076e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:13.182086 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcd483-f523-4fe3-9188-1d6564a076e4 None admin] RBAC: Adding query filter params (interface=public, service_id=ebdcd5f17cf34db384dce4d46e3a0488, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:24:13.182400 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcd483-f523-4fe3-9188-1d6564a076e4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:13.183796 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcd483-f523-4fe3-9188-1d6564a076e4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:13.188724 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 650/1300] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Jun 29 12:24:13 2022] GET /identity/v3/endpoints?service_id=ebdcd5f17cf34db384dce4d46e3a0488&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:24:13.195046 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3353f8c-78b9-4240-8685-3edbc1c57919 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:13.274837 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:24:13.292487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3353f8c-78b9-4240-8685-3edbc1c57919 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:13.334238 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62bc78-22d7-4d74-b385-a3a83720667a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:13.334957 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62bc78-22d7-4d74-b385-a3a83720667a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:13.335249 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62bc78-22d7-4d74-b385-a3a83720667a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:13.335534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62bc78-22d7-4d74-b385-a3a83720667a None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:13.336219 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62bc78-22d7-4d74-b385-a3a83720667a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:13.337589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62bc78-22d7-4d74-b385-a3a83720667a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:13.344431 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 651/1301] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Jun 29 12:24:13 2022] GET /identity/v3/services => generated 2191 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:24:15.333881 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e109b-acb3-4b36-b1ff-b7c455e20bca None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:15.334475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e109b-acb3-4b36-b1ff-b7c455e20bca None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:15.334475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e109b-acb3-4b36-b1ff-b7c455e20bca None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:15.335842 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 651/1302] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:24:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:24:15.340942 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93385019-86fd-4d3a-8a32-981bd7f34d85 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:15.341350 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93385019-86fd-4d3a-8a32-981bd7f34d85 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:15.341530 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93385019-86fd-4d3a-8a32-981bd7f34d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:15.358324 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-93385019-86fd-4d3a-8a32-981bd7f34d85 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:24:15.359540 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-93385019-86fd-4d3a-8a32-981bd7f34d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:24:15.423823 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 652/1303] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:24:15 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 29 12:24:15.447388 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e109b-acb3-4b36-b1ff-b7c455e20bca None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:15.538577 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:24:15.545284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e109b-acb3-4b36-b1ff-b7c455e20bca None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:15.577488 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9998e1-385e-469d-b63c-c311fd637046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:15.578262 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9998e1-385e-469d-b63c-c311fd637046 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:15.578650 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9998e1-385e-469d-b63c-c311fd637046 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:15.578929 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9998e1-385e-469d-b63c-c311fd637046 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:15.579574 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9998e1-385e-469d-b63c-c311fd637046 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:15.580923 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9998e1-385e-469d-b63c-c311fd637046 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:15.592494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9b9998e1-385e-469d-b63c-c311fd637046 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 12:24:15.592494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:24:15.592494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 29 12:24:15.592494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 29 12:24:15.592494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 29 12:24:15.592494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 29 12:24:15.592494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 29 12:24:15.592494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:24:15.592494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 12:24:15.592494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:24:15.592494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 29 12:24:15.592494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:24:15.592494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 29 12:24:15.592494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 29 12:24:15.592494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 29 12:24:15.592494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 29 12:24:15.592494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 29 12:24:15.593714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 29 12:24:15.593714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 29 12:24:15.593714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 29 12:24:15.593714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 29 12:24:15.593714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 29 12:24:15.593714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 29 12:24:15.593714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 29 12:24:15.593714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 29 12:24:15.593714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 29 12:24:15.593714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 29 12:24:15.593714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 29 12:24:15.593714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 29 12:24:15.593714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 29 12:24:15.593714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 29 12:24:15.593714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 29 12:24:15.593714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jun 29 12:24:15.594628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 29 12:24:15.594628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jun 29 12:24:15.594628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 29 12:24:15.594628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jun 29 12:24:15.594628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 29 12:24:15.594628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 29 12:24:15.594628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 29 12:24:15.594628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jun 29 12:24:15.594628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 29 12:24:15.594628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jun 29 12:24:15.594628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 29 12:24:15.594628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 29 12:24:15.594628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 29 12:24:15.594628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jun 29 12:24:15.594628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 652/1304] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jun 29 12:24:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 29 12:24:15.599760 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93385019-86fd-4d3a-8a32-981bd7f34d85 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:15.674142 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:24:15.697082 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93385019-86fd-4d3a-8a32-981bd7f34d85 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:15.729563 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e426e2-4cec-4b01-9a4d-5abe4ad32516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:15.730219 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e426e2-4cec-4b01-9a4d-5abe4ad32516 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:15.730533 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e426e2-4cec-4b01-9a4d-5abe4ad32516 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:15.730812 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e426e2-4cec-4b01-9a4d-5abe4ad32516 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:15.731522 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e426e2-4cec-4b01-9a4d-5abe4ad32516 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:24:15.731829 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e426e2-4cec-4b01-9a4d-5abe4ad32516 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:15.733181 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e426e2-4cec-4b01-9a4d-5abe4ad32516 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:15.738178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 653/1305] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Jun 29 12:24:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:24:15.744419 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9998e1-385e-469d-b63c-c311fd637046 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:24:15.839569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9998e1-385e-469d-b63c-c311fd637046 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:24:15.871903 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2241a9d1-23de-4f57-9b95-ac751d64949c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:24:15.872489 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241a9d1-23de-4f57-9b95-ac751d64949c None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:24:15.872772 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241a9d1-23de-4f57-9b95-ac751d64949c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:24:15.873048 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2241a9d1-23de-4f57-9b95-ac751d64949c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:24:15.873783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2241a9d1-23de-4f57-9b95-ac751d64949c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:24:15.875159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2241a9d1-23de-4f57-9b95-ac751d64949c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:24:15.892377 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 653/1306] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Jun 29 12:24:15 2022] POST /identity/v3/endpoints => generated 331 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:24:25.735372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=30 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:24:25.888600 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:25:18.502077 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2c904-a5c8-4e18-a208-b6fb1f4c6c7e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:18.502742 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2c904-a5c8-4e18-a208-b6fb1f4c6c7e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:18.502742 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2c904-a5c8-4e18-a208-b6fb1f4c6c7e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:18.509976 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dccb68-782b-4f4d-a904-8cbd5a20229d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:18.510460 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dccb68-782b-4f4d-a904-8cbd5a20229d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:18.510552 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dccb68-782b-4f4d-a904-8cbd5a20229d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:18.513983 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 654/1307] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:25:18 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:25:18.590178 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f8dccb68-782b-4f4d-a904-8cbd5a20229d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:25:18.591437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f8dccb68-782b-4f4d-a904-8cbd5a20229d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:18.718817 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 654/1308] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:25:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:25:18.749536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6dd58-488c-40aa-b7b3-6eda392c3d85 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:18.750077 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6dd58-488c-40aa-b7b3-6eda392c3d85 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:18.750728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6dd58-488c-40aa-b7b3-6eda392c3d85 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:18.753617 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 655/1309] 10.222.0.52 () {54 vars in 964 bytes} [Wed Jun 29 12:25:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:25:18.759691 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d934d-1cec-426c-ae21-ac766d14b514 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:18.760094 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d934d-1cec-426c-ae21-ac766d14b514 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:18.760263 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d934d-1cec-426c-ae21-ac766d14b514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:18.860172 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-514d934d-1cec-426c-ae21-ac766d14b514 None None] MFA Rules not processed for user `ce922cdbf2d34f10b6e49631f0c6d770`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:25:18.861414 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-514d934d-1cec-426c-ae21-ac766d14b514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:18.892476 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:25:19.003773 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6dd58-488c-40aa-b7b3-6eda392c3d85 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:25:19.006128 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 655/1310] 10.222.0.52 () {58 vars in 1061 bytes} [Wed Jun 29 12:25:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:25:19.007140 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7dd6dd58-488c-40aa-b7b3-6eda392c3d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:19.173497 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6dd58-488c-40aa-b7b3-6eda392c3d85 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:25:19.253602 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2cffd6-d93f-4a28-8cce-5a3f75929cf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:25:19.254304 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2cffd6-d93f-4a28-8cce-5a3f75929cf5 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:19.254682 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2cffd6-d93f-4a28-8cce-5a3f75929cf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:19.254908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2cffd6-d93f-4a28-8cce-5a3f75929cf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:19.256526 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fd2cffd6-d93f-4a28-8cce-5a3f75929cf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:19.315438 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:25:19.385493 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2cffd6-d93f-4a28-8cce-5a3f75929cf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:25:19.387784 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2cffd6-d93f-4a28-8cce-5a3f75929cf5 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:25:19.411899 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 656/1311] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:25:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 12:25:21.790764 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0591461-eec3-4710-881a-57c6712fe296 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:21.791295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0591461-eec3-4710-881a-57c6712fe296 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:21.791408 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0591461-eec3-4710-881a-57c6712fe296 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:21.792702 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 656/1312] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:25:21.798038 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273e942-e794-4d1d-8043-08df67591175 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:21.798465 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273e942-e794-4d1d-8043-08df67591175 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:21.798732 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0273e942-e794-4d1d-8043-08df67591175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:21.892670 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0273e942-e794-4d1d-8043-08df67591175 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:25:21.894241 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0273e942-e794-4d1d-8043-08df67591175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:21.990076 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 657/1313] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:25:22.043492 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1268c4-b64d-43e2-b2a1-f7f48ae6ae5a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:22.043690 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1268c4-b64d-43e2-b2a1-f7f48ae6ae5a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:22.043981 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1268c4-b64d-43e2-b2a1-f7f48ae6ae5a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:22.045355 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 657/1314] 10.222.0.52 () {54 vars in 964 bytes} [Wed Jun 29 12:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:25:22.051992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce19d9-4b6d-40ef-837d-a9cb0ab040be None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:22.052283 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce19d9-4b6d-40ef-837d-a9cb0ab040be None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:22.052565 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce19d9-4b6d-40ef-837d-a9cb0ab040be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:22.062984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:25:22.128879 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bfce19d9-4b6d-40ef-837d-a9cb0ab040be None None] MFA Rules not processed for user `ce922cdbf2d34f10b6e49631f0c6d770`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:25:22.130186 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bfce19d9-4b6d-40ef-837d-a9cb0ab040be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:22.214094 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 658/1315] 10.222.0.52 () {58 vars in 1061 bytes} [Wed Jun 29 12:25:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:25:22.224537 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1268c4-b64d-43e2-b2a1-f7f48ae6ae5a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:25:22.226081 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ae1268c4-b64d-43e2-b2a1-f7f48ae6ae5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:22.338841 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1268c4-b64d-43e2-b2a1-f7f48ae6ae5a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:25:22.416221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:25:22.419145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c51091d-dbb5-4734-a2ca-f49090e7791d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:25:22.420181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51091d-dbb5-4734-a2ca-f49090e7791d service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:22.420484 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51091d-dbb5-4734-a2ca-f49090e7791d service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:22.420916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c51091d-dbb5-4734-a2ca-f49090e7791d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:22.422780 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7c51091d-dbb5-4734-a2ca-f49090e7791d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:22.532302 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51091d-dbb5-4734-a2ca-f49090e7791d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:25:22.534252 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c51091d-dbb5-4734-a2ca-f49090e7791d service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:25:22.553587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 658/1316] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:25:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:25:25.223751 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f463cc2-6d2c-4c0d-8bc6-884990fbf2ff None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:25.224318 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f463cc2-6d2c-4c0d-8bc6-884990fbf2ff None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:25.224318 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f463cc2-6d2c-4c0d-8bc6-884990fbf2ff None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:25.225668 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 659/1317] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:25:25.230845 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d22b54-7bef-4ae6-a953-208dadde25c5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:25.231246 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d22b54-7bef-4ae6-a953-208dadde25c5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:25.231404 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d22b54-7bef-4ae6-a953-208dadde25c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:25.296397 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c2d22b54-7bef-4ae6-a953-208dadde25c5 None None] MFA Rules not processed for user `3f6a358f4bc044be94e44d1470a32c67`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:25:25.297579 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c2d22b54-7bef-4ae6-a953-208dadde25c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:25.349576 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:25:25.394482 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 659/1318] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:25:25 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:25:25.434283 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f463cc2-6d2c-4c0d-8bc6-884990fbf2ff None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:25:25.435850 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9f463cc2-6d2c-4c0d-8bc6-884990fbf2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:25.536750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f463cc2-6d2c-4c0d-8bc6-884990fbf2ff None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:25:25.539564 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:25:25.610838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f974520-0666-4019-8d72-18b8d57b8759 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:25:25.611483 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f974520-0666-4019-8d72-18b8d57b8759 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:25.612298 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f974520-0666-4019-8d72-18b8d57b8759 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:25.612595 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f974520-0666-4019-8d72-18b8d57b8759 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:25.614206 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4f974520-0666-4019-8d72-18b8d57b8759 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:25.720871 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f974520-0666-4019-8d72-18b8d57b8759 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:25:25.722620 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f974520-0666-4019-8d72-18b8d57b8759 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:25:25.741024 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 660/1319] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:25:25 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:25:28.444867 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719a9bc-42be-4960-ac5f-21973663ec4f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:28.445383 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719a9bc-42be-4960-ac5f-21973663ec4f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:28.445481 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a719a9bc-42be-4960-ac5f-21973663ec4f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:28.446824 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 660/1320] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:25:28.452032 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fbb51-70a7-4ee6-a93c-addbc6fb8220 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:28.452426 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fbb51-70a7-4ee6-a93c-addbc6fb8220 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:28.452606 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fbb51-70a7-4ee6-a93c-addbc6fb8220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:28.491954 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:25:28.516781 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-665fbb51-70a7-4ee6-a93c-addbc6fb8220 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:25:28.517995 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-665fbb51-70a7-4ee6-a93c-addbc6fb8220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:28.610052 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 661/1321] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:25:28 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:25:28.651872 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719a9bc-42be-4960-ac5f-21973663ec4f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:25:28.653375 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a719a9bc-42be-4960-ac5f-21973663ec4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:28.754286 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a719a9bc-42be-4960-ac5f-21973663ec4f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:25:28.824641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2512def-5167-4c6f-a54b-d3ef24a17969 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:25:28.825276 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2512def-5167-4c6f-a54b-d3ef24a17969 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:28.825558 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2512def-5167-4c6f-a54b-d3ef24a17969 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:28.825833 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2512def-5167-4c6f-a54b-d3ef24a17969 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:28.827536 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f2512def-5167-4c6f-a54b-d3ef24a17969 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:28.834608 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:25:28.934279 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2512def-5167-4c6f-a54b-d3ef24a17969 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:25:28.935682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2512def-5167-4c6f-a54b-d3ef24a17969 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:25:28.953280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 661/1322] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:25:28 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:25:31.741922 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb084ce-91d6-4407-97eb-f5c79ed6d682 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:31.742475 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb084ce-91d6-4407-97eb-f5c79ed6d682 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:31.742726 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb084ce-91d6-4407-97eb-f5c79ed6d682 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:31.744188 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 662/1323] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:25:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:25:31.749537 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a45921-bd8a-4efe-8c35-3ebfca5213dd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:31.749962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a45921-bd8a-4efe-8c35-3ebfca5213dd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:31.750220 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a45921-bd8a-4efe-8c35-3ebfca5213dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:31.817303 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f3a45921-bd8a-4efe-8c35-3ebfca5213dd None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:25:31.818544 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f3a45921-bd8a-4efe-8c35-3ebfca5213dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:31.882622 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:25:31.929268 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 662/1324] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:25:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:25:31.967779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb084ce-91d6-4407-97eb-f5c79ed6d682 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:25:31.969328 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-acb084ce-91d6-4407-97eb-f5c79ed6d682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:32.078216 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb084ce-91d6-4407-97eb-f5c79ed6d682 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:25:32.092568 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:25:32.157584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67eba5-10a9-491d-9fa5-770253b14a4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:25:32.158239 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67eba5-10a9-491d-9fa5-770253b14a4d service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:32.158600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67eba5-10a9-491d-9fa5-770253b14a4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:32.158992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67eba5-10a9-491d-9fa5-770253b14a4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:32.160657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ce67eba5-10a9-491d-9fa5-770253b14a4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:32.273829 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce67eba5-10a9-491d-9fa5-770253b14a4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:25:32.275189 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce67eba5-10a9-491d-9fa5-770253b14a4d service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:25:32.293508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 663/1325] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:25:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:25:34.385834 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd91ee-de8a-4ef2-ac93-ea729478c1d6 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:34.386310 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd91ee-de8a-4ef2-ac93-ea729478c1d6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:34.386442 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd91ee-de8a-4ef2-ac93-ea729478c1d6 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:34.387797 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 663/1326] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:25:34.393607 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f67cb-a4be-4776-a99f-c4f293e4adbc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:34.394218 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f67cb-a4be-4776-a99f-c4f293e4adbc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:34.394536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f67cb-a4be-4776-a99f-c4f293e4adbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:34.442391 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:25:34.462010 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a99f67cb-a4be-4776-a99f-c4f293e4adbc None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:25:34.463328 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a99f67cb-a4be-4776-a99f-c4f293e4adbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:34.561366 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 664/1327] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:25:34.596977 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd91ee-de8a-4ef2-ac93-ea729478c1d6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:25:34.598649 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a3dd91ee-de8a-4ef2-ac93-ea729478c1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:34.704368 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dd91ee-de8a-4ef2-ac93-ea729478c1d6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:25:34.780559 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b790b2-6012-44e3-a27f-051c31a7f95f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:25:34.781200 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b790b2-6012-44e3-a27f-051c31a7f95f service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:34.781483 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b790b2-6012-44e3-a27f-051c31a7f95f service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:34.781762 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b790b2-6012-44e3-a27f-051c31a7f95f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:34.783912 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-37b790b2-6012-44e3-a27f-051c31a7f95f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:34.800941 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:25:34.892649 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b790b2-6012-44e3-a27f-051c31a7f95f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:25:34.894299 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b790b2-6012-44e3-a27f-051c31a7f95f service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:25:34.913134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 664/1328] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:25:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:25:37.037765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2167310-6765-48d8-a303-b7f4749cd537 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:37.038369 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2167310-6765-48d8-a303-b7f4749cd537 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:37.038369 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2167310-6765-48d8-a303-b7f4749cd537 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:37.039740 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 665/1329] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:25:37.045158 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01073b-ff30-4645-807c-3492e76828d8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:37.045610 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01073b-ff30-4645-807c-3492e76828d8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:37.045712 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01073b-ff30-4645-807c-3492e76828d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:37.117773 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5b01073b-ff30-4645-807c-3492e76828d8 None None] MFA Rules not processed for user `3f6a358f4bc044be94e44d1470a32c67`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:25:37.119019 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5b01073b-ff30-4645-807c-3492e76828d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:37.196656 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:25:37.230857 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 665/1330] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:25:37 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:25:37.269681 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2167310-6765-48d8-a303-b7f4749cd537 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:25:37.271293 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b2167310-6765-48d8-a303-b7f4749cd537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:37.380644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2167310-6765-48d8-a303-b7f4749cd537 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:25:37.401276 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:25:37.454379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b2e210-a5fd-4a6f-b66b-b2ea4b5cd8c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:25:37.455049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2e210-a5fd-4a6f-b66b-b2ea4b5cd8c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:37.455860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2e210-a5fd-4a6f-b66b-b2ea4b5cd8c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:37.456116 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2e210-a5fd-4a6f-b66b-b2ea4b5cd8c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:37.457732 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-85b2e210-a5fd-4a6f-b66b-b2ea4b5cd8c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:37.578191 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b2e210-a5fd-4a6f-b66b-b2ea4b5cd8c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:25:37.579673 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b2e210-a5fd-4a6f-b66b-b2ea4b5cd8c4 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:25:37.599627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 666/1331] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:25:37 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:25:40.050633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13076d78-58b4-4af1-9db9-2777872ee3db None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:40.051206 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13076d78-58b4-4af1-9db9-2777872ee3db None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:40.051206 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13076d78-58b4-4af1-9db9-2777872ee3db None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:40.052587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 666/1332] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:25:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:25:40.057886 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082e3ee-8e6c-47d5-bf4c-504c3b9350cc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:40.058351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082e3ee-8e6c-47d5-bf4c-504c3b9350cc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:40.058514 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082e3ee-8e6c-47d5-bf4c-504c3b9350cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:40.122179 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:25:40.125271 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c082e3ee-8e6c-47d5-bf4c-504c3b9350cc None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:25:40.127854 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c082e3ee-8e6c-47d5-bf4c-504c3b9350cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:40.232094 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 667/1333] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:25:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:25:40.269736 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13076d78-58b4-4af1-9db9-2777872ee3db None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:25:40.271454 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-13076d78-58b4-4af1-9db9-2777872ee3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:40.383009 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13076d78-58b4-4af1-9db9-2777872ee3db None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:25:40.458365 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4a777-0af7-4a18-90e0-93d2d17fc6e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:25:40.459031 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4a777-0af7-4a18-90e0-93d2d17fc6e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:40.459312 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4a777-0af7-4a18-90e0-93d2d17fc6e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:40.459589 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4a777-0af7-4a18-90e0-93d2d17fc6e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:40.461230 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7fd4a777-0af7-4a18-90e0-93d2d17fc6e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:40.488272 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:25:40.570693 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4a777-0af7-4a18-90e0-93d2d17fc6e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:25:40.572130 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4a777-0af7-4a18-90e0-93d2d17fc6e4 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:25:40.589849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 667/1334] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:25:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:25:43.021393 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efff75-c935-425d-83f4-df805f3b2e4f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:43.021935 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efff75-c935-425d-83f4-df805f3b2e4f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:43.022085 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efff75-c935-425d-83f4-df805f3b2e4f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:43.024021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 668/1335] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:25:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:25:43.029289 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacf947-d4aa-4a76-931e-fcdbdfcd6c5f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:43.029728 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacf947-d4aa-4a76-931e-fcdbdfcd6c5f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:43.029863 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eacf947-d4aa-4a76-931e-fcdbdfcd6c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:43.093737 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8eacf947-d4aa-4a76-931e-fcdbdfcd6c5f None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:25:43.094953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8eacf947-d4aa-4a76-931e-fcdbdfcd6c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:43.172720 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:25:43.195956 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 668/1336] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:25:43.239671 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efff75-c935-425d-83f4-df805f3b2e4f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:25:43.241180 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-71efff75-c935-425d-83f4-df805f3b2e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:43.350654 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efff75-c935-425d-83f4-df805f3b2e4f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:25:43.382246 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:25:43.431366 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c819f-d08d-4d54-8333-b6d85fdf7fdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:25:43.432065 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c819f-d08d-4d54-8333-b6d85fdf7fdf service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:43.432372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c819f-d08d-4d54-8333-b6d85fdf7fdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:43.432686 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c819f-d08d-4d54-8333-b6d85fdf7fdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:43.434484 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ea7c819f-d08d-4d54-8333-b6d85fdf7fdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:43.551119 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c819f-d08d-4d54-8333-b6d85fdf7fdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:25:43.552661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c819f-d08d-4d54-8333-b6d85fdf7fdf service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:25:43.572278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 669/1337] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:25:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:25:45.968132 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd293b0-6916-4326-b6f9-bacb0c8f992c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:45.968670 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd293b0-6916-4326-b6f9-bacb0c8f992c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:45.968670 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd293b0-6916-4326-b6f9-bacb0c8f992c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:45.970053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 669/1338] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:25:45.975879 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b45d7-c00e-4e17-a1a6-d435ed1eeab7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:45.976310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b45d7-c00e-4e17-a1a6-d435ed1eeab7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:45.976447 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b45d7-c00e-4e17-a1a6-d435ed1eeab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:46.040559 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-540b45d7-c00e-4e17-a1a6-d435ed1eeab7 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:25:46.041751 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-540b45d7-c00e-4e17-a1a6-d435ed1eeab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:46.050028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:25:46.149520 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 670/1339] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:25:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:25:46.184862 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd293b0-6916-4326-b6f9-bacb0c8f992c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:25:46.186368 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4cd293b0-6916-4326-b6f9-bacb0c8f992c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:46.287762 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd293b0-6916-4326-b6f9-bacb0c8f992c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:25:46.358940 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31e5ea9-4123-4880-b160-cd8fee2cd8b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:25:46.359692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e5ea9-4123-4880-b160-cd8fee2cd8b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:46.359976 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e5ea9-4123-4880-b160-cd8fee2cd8b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:46.360250 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31e5ea9-4123-4880-b160-cd8fee2cd8b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:46.361976 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c31e5ea9-4123-4880-b160-cd8fee2cd8b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:46.398414 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:25:46.470673 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31e5ea9-4123-4880-b160-cd8fee2cd8b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:25:46.472102 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31e5ea9-4123-4880-b160-cd8fee2cd8b8 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:25:46.490182 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 670/1340] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:25:46 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:25:50.488133 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedcb7d-3d66-4aa5-b42e-b6e9fa14b862 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:50.488693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedcb7d-3d66-4aa5-b42e-b6e9fa14b862 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:50.488693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedcb7d-3d66-4aa5-b42e-b6e9fa14b862 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:50.489977 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 671/1341] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:25:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:25:50.495379 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486c43c-4aea-4f23-b58a-c5f410dc7b37 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:50.495869 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486c43c-4aea-4f23-b58a-c5f410dc7b37 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:50.495976 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486c43c-4aea-4f23-b58a-c5f410dc7b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:50.567910 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4486c43c-4aea-4f23-b58a-c5f410dc7b37 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:25:50.569097 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4486c43c-4aea-4f23-b58a-c5f410dc7b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:50.662013 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:25:50.675951 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 671/1342] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:25:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:25:50.714583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dedcb7d-3d66-4aa5-b42e-b6e9fa14b862 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:25:50.716743 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5dedcb7d-3d66-4aa5-b42e-b6e9fa14b862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:50.813166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dedcb7d-3d66-4aa5-b42e-b6e9fa14b862 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:25:50.817380 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b24fa9c-bb42-4f3e-888f-798db21c9481 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:25:50.818029 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24fa9c-bb42-4f3e-888f-798db21c9481 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:50.818315 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24fa9c-bb42-4f3e-888f-798db21c9481 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:50.818621 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b24fa9c-bb42-4f3e-888f-798db21c9481 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:50.827827 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b24fa9c-bb42-4f3e-888f-798db21c9481 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:25:50.829264 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b24fa9c-bb42-4f3e-888f-798db21c9481 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:25:50.839545 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 672/1343] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:25:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:25:53.620049 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e443174-3de5-41d5-ac9e-32bdea64241f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:53.620569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e443174-3de5-41d5-ac9e-32bdea64241f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:53.620569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e443174-3de5-41d5-ac9e-32bdea64241f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:53.622989 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 672/1344] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:25:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:25:53.629055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf23b0e-4a8d-477d-99fc-ebe41cbb107a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:53.629590 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf23b0e-4a8d-477d-99fc-ebe41cbb107a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:53.629682 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf23b0e-4a8d-477d-99fc-ebe41cbb107a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:53.660459 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2bf23b0e-4a8d-477d-99fc-ebe41cbb107a None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:25:53.661728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2bf23b0e-4a8d-477d-99fc-ebe41cbb107a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:53.692313 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:25:53.745318 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 673/1345] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:25:53 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:25:53.782735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e443174-3de5-41d5-ac9e-32bdea64241f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:25:53.793896 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e443174-3de5-41d5-ac9e-32bdea64241f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:25:53.797338 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ad01b7-bf20-470d-ae95-a2cd4d4851b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:25:53.797968 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad01b7-bf20-470d-ae95-a2cd4d4851b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:53.798250 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad01b7-bf20-470d-ae95-a2cd4d4851b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:53.798611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ad01b7-bf20-470d-ae95-a2cd4d4851b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:53.808057 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad01b7-bf20-470d-ae95-a2cd4d4851b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:25:53.809482 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ad01b7-bf20-470d-ae95-a2cd4d4851b6 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:25:53.819074 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 673/1346] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:25:53 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:25:58.073797 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d687bc-4583-4047-a88d-d8892d84718e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:58.074325 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d687bc-4583-4047-a88d-d8892d84718e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:58.074616 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d687bc-4583-4047-a88d-d8892d84718e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:58.075750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 674/1347] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:25:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:25:58.081223 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f35e7d8-a441-4d46-9d8f-6e596b24e50f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:58.081714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f35e7d8-a441-4d46-9d8f-6e596b24e50f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:58.081832 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f35e7d8-a441-4d46-9d8f-6e596b24e50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:58.100748 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1f35e7d8-a441-4d46-9d8f-6e596b24e50f None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:25:58.101877 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1f35e7d8-a441-4d46-9d8f-6e596b24e50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:58.109705 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 674/1348] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:25:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:25:58.147980 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d687bc-4583-4047-a88d-d8892d84718e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:25:58.149695 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-50d687bc-4583-4047-a88d-d8892d84718e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:25:58.163334 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d687bc-4583-4047-a88d-d8892d84718e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:25:58.166771 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b49f7b3-a1c6-4510-965b-62180ade862b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:25:58.167413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b49f7b3-a1c6-4510-965b-62180ade862b service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:25:58.167700 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b49f7b3-a1c6-4510-965b-62180ade862b service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:25:58.167978 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b49f7b3-a1c6-4510-965b-62180ade862b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:25:58.177549 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b49f7b3-a1c6-4510-965b-62180ade862b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:25:58.179043 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b49f7b3-a1c6-4510-965b-62180ade862b service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:25:58.188502 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 675/1349] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:25:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:26:03.252620 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53634f5-7015-4943-bcb3-1cf2e9d9df30 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:03.253122 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53634f5-7015-4943-bcb3-1cf2e9d9df30 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:03.253410 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53634f5-7015-4943-bcb3-1cf2e9d9df30 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:03.255372 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 675/1350] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jun 29 12:26:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:26:03.263565 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05068626-0a39-44fa-9a0b-29c381eaf4e6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:03.264342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05068626-0a39-44fa-9a0b-29c381eaf4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:03.264772 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05068626-0a39-44fa-9a0b-29c381eaf4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:03.341532 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-05068626-0a39-44fa-9a0b-29c381eaf4e6 None None] MFA Rules not processed for user `c98135cacb9d4f22846ad33500b9bd1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:26:03.349948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-05068626-0a39-44fa-9a0b-29c381eaf4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:26:03.434967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 676/1351] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jun 29 12:26:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:26:07.572122 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff5d9b-20db-430d-9f2d-9142b69431c3 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:07.572690 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff5d9b-20db-430d-9f2d-9142b69431c3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:07.573237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff5d9b-20db-430d-9f2d-9142b69431c3 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:07.574719 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 676/1352] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jun 29 12:26:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:26:07.580624 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524fdc1-858f-4049-8013-76af6e1129fc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:07.581158 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524fdc1-858f-4049-8013-76af6e1129fc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:07.581245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d524fdc1-858f-4049-8013-76af6e1129fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:07.599885 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d524fdc1-858f-4049-8013-76af6e1129fc None None] MFA Rules not processed for user `c98135cacb9d4f22846ad33500b9bd1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:26:07.601070 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d524fdc1-858f-4049-8013-76af6e1129fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:26:07.609729 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 677/1353] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jun 29 12:26:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:26:08.095532 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=13 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:26:09.240506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d1a8c-0b25-40eb-a7af-6fdd1fb9be08 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:09.241160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d1a8c-0b25-40eb-a7af-6fdd1fb9be08 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:09.241160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898d1a8c-0b25-40eb-a7af-6fdd1fb9be08 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:09.242439 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 677/1354] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jun 29 12:26:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:26:09.247287 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ea6a2-05ac-40e9-ae3b-212ec41b5834 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:09.247686 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ea6a2-05ac-40e9-ae3b-212ec41b5834 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:09.247871 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ea6a2-05ac-40e9-ae3b-212ec41b5834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:09.266107 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-be2ea6a2-05ac-40e9-ae3b-212ec41b5834 None None] MFA Rules not processed for user `c98135cacb9d4f22846ad33500b9bd1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:26:09.268581 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-be2ea6a2-05ac-40e9-ae3b-212ec41b5834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:26:09.277405 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 678/1355] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jun 29 12:26:09 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:26:12.783474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4af7d-ff41-4b7e-85a2-f6c131b0ea3f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:12.784463 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4af7d-ff41-4b7e-85a2-f6c131b0ea3f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:12.784759 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4af7d-ff41-4b7e-85a2-f6c131b0ea3f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:12.786317 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 678/1356] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:26:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:26:12.793008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdc72c-833a-42fc-9040-777224b20500 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:12.795299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdc72c-833a-42fc-9040-777224b20500 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:12.795805 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdc72c-833a-42fc-9040-777224b20500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:12.817256 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c5fdc72c-833a-42fc-9040-777224b20500 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:26:12.819219 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c5fdc72c-833a-42fc-9040-777224b20500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:26:12.827421 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 679/1357] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:26:12.883009 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d552a9c-edb6-401a-ab01-a6300f1a8be0 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:12.883467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d552a9c-edb6-401a-ab01-a6300f1a8be0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:12.883972 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d552a9c-edb6-401a-ab01-a6300f1a8be0 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:12.885557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 679/1358] 10.222.0.52 () {54 vars in 961 bytes} [Wed Jun 29 12:26:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:26:12.899269 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41396fe-8405-4335-b9f2-9fa0d4e537cf None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:12.900212 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41396fe-8405-4335-b9f2-9fa0d4e537cf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:12.900786 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41396fe-8405-4335-b9f2-9fa0d4e537cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:12.945516 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:26:12.955255 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e41396fe-8405-4335-b9f2-9fa0d4e537cf None None] MFA Rules not processed for user `3ac7391a3c41499d94e5f97478797313`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:26:12.956472 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e41396fe-8405-4335-b9f2-9fa0d4e537cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:26:13.042024 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 680/1359] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Jun 29 12:26:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:26:13.051617 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d552a9c-edb6-401a-ab01-a6300f1a8be0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:26:13.066673 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d552a9c-edb6-401a-ab01-a6300f1a8be0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:26:13.072131 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f5eba4-3264-468d-9779-f13df3de54e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:26:13.072839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5eba4-3264-468d-9779-f13df3de54e7 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:13.073330 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5eba4-3264-468d-9779-f13df3de54e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:13.073679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5eba4-3264-468d-9779-f13df3de54e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:13.083256 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f5eba4-3264-468d-9779-f13df3de54e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:26:13.084941 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f5eba4-3264-468d-9779-f13df3de54e7 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:26:13.095312 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 680/1360] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:26:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:26:15.362386 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78969d-7959-415f-99aa-4517a8c2fbc0 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:15.363315 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78969d-7959-415f-99aa-4517a8c2fbc0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:15.364249 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78969d-7959-415f-99aa-4517a8c2fbc0 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:15.365627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 681/1361] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:26:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:26:15.371106 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c0c67-b0cc-45b5-a09b-2dc75cc89c50 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:15.371559 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c0c67-b0cc-45b5-a09b-2dc75cc89c50 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:15.371672 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c0c67-b0cc-45b5-a09b-2dc75cc89c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:15.390006 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7c7c0c67-b0cc-45b5-a09b-2dc75cc89c50 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:26:15.391234 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7c7c0c67-b0cc-45b5-a09b-2dc75cc89c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:26:15.400041 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 681/1362] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:26:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:26:15.445465 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd78969d-7959-415f-99aa-4517a8c2fbc0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:26:15.460809 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd78969d-7959-415f-99aa-4517a8c2fbc0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:26:15.466186 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b596992-71d0-489a-8abd-fce374426d24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:26:15.466873 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b596992-71d0-489a-8abd-fce374426d24 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:15.467165 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b596992-71d0-489a-8abd-fce374426d24 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:15.467471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b596992-71d0-489a-8abd-fce374426d24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:15.476417 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b596992-71d0-489a-8abd-fce374426d24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:26:15.478016 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b596992-71d0-489a-8abd-fce374426d24 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:26:15.488232 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 682/1363] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:26:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:26:17.863119 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e075983-1630-472d-925a-9dfe7b83accf None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:17.863613 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e075983-1630-472d-925a-9dfe7b83accf None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:17.863613 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e075983-1630-472d-925a-9dfe7b83accf None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:17.865314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 682/1364] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:26:17.877702 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b822acc-9e48-4754-ac78-e29b4737b703 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:17.878125 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b822acc-9e48-4754-ac78-e29b4737b703 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:17.878290 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b822acc-9e48-4754-ac78-e29b4737b703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:17.907159 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3b822acc-9e48-4754-ac78-e29b4737b703 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:26:17.909617 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3b822acc-9e48-4754-ac78-e29b4737b703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:26:17.925298 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 683/1365] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:26:17.976820 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9224ea-8f8b-45f0-a0ee-65ba59f065ae None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:17.977111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9224ea-8f8b-45f0-a0ee-65ba59f065ae None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:17.977389 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9224ea-8f8b-45f0-a0ee-65ba59f065ae None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:17.978838 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 683/1366] 10.222.0.52 () {54 vars in 961 bytes} [Wed Jun 29 12:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:26:17.984236 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2511bc-dc88-4a67-9739-9a561fd5bd50 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:17.984526 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2511bc-dc88-4a67-9739-9a561fd5bd50 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:17.984805 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2511bc-dc88-4a67-9739-9a561fd5bd50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:18.003057 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4b2511bc-dc88-4a67-9739-9a561fd5bd50 None None] MFA Rules not processed for user `3ac7391a3c41499d94e5f97478797313`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:26:18.004227 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4b2511bc-dc88-4a67-9739-9a561fd5bd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:26:18.013274 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 684/1367] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Jun 29 12:26:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:26:18.021753 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9224ea-8f8b-45f0-a0ee-65ba59f065ae None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:26:18.036820 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9224ea-8f8b-45f0-a0ee-65ba59f065ae None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:26:18.042087 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5498a9-3823-4806-a1c8-8b2721919935 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:26:18.042816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5498a9-3823-4806-a1c8-8b2721919935 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:18.043101 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5498a9-3823-4806-a1c8-8b2721919935 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:18.043377 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5498a9-3823-4806-a1c8-8b2721919935 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:18.051821 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5498a9-3823-4806-a1c8-8b2721919935 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:26:18.053419 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5498a9-3823-4806-a1c8-8b2721919935 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:26:18.064047 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 684/1368] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:26:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:26:20.175753 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0fbaf-3f18-49dc-8e1e-72ee37581241 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:20.176303 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0fbaf-3f18-49dc-8e1e-72ee37581241 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:20.176303 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0fbaf-3f18-49dc-8e1e-72ee37581241 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:20.178477 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 685/1369] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:26:20.183436 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78fa1b-0816-406e-8773-a3491b5a38b0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:20.183842 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78fa1b-0816-406e-8773-a3491b5a38b0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:20.184005 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78fa1b-0816-406e-8773-a3491b5a38b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:20.202003 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9b78fa1b-0816-406e-8773-a3491b5a38b0 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:26:20.203222 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9b78fa1b-0816-406e-8773-a3491b5a38b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:26:20.212164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 685/1370] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:26:20.257607 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0fbaf-3f18-49dc-8e1e-72ee37581241 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:26:20.273719 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0fbaf-3f18-49dc-8e1e-72ee37581241 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:26:20.279252 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df4c474-4a4c-40cb-992b-0326ce3d8bc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:26:20.280003 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4c474-4a4c-40cb-992b-0326ce3d8bc0 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:20.280325 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4c474-4a4c-40cb-992b-0326ce3d8bc0 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:20.280599 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4c474-4a4c-40cb-992b-0326ce3d8bc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:20.289388 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df4c474-4a4c-40cb-992b-0326ce3d8bc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:26:20.291045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df4c474-4a4c-40cb-992b-0326ce3d8bc0 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:26:20.301514 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 686/1371] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:26:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:26:20.948913 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12610a4b-fc69-4f62-82e0-34e0c0535705 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:20.949203 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12610a4b-fc69-4f62-82e0-34e0c0535705 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:20.949761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12610a4b-fc69-4f62-82e0-34e0c0535705 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:20.951962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 686/1372] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:26:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:26:20.958372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc3fcf-30bb-424f-ae02-d558d57734b2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:20.958735 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc3fcf-30bb-424f-ae02-d558d57734b2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:20.959226 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc3fcf-30bb-424f-ae02-d558d57734b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:20.978071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-07fc3fcf-30bb-424f-ae02-d558d57734b2 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:26:20.979293 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-07fc3fcf-30bb-424f-ae02-d558d57734b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:26:20.987640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 687/1373] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:26:21.333902 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783462b-802d-42fe-a37f-4e92a32bd59b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:21.333902 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783462b-802d-42fe-a37f-4e92a32bd59b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:21.333902 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783462b-802d-42fe-a37f-4e92a32bd59b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:21.337239 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 687/1374] 10.222.0.52 () {54 vars in 963 bytes} [Wed Jun 29 12:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:26:21.340006 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f2da3-b977-487d-8e69-d4258d31629d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:21.340006 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f2da3-b977-487d-8e69-d4258d31629d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:21.340576 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f2da3-b977-487d-8e69-d4258d31629d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:21.396397 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8e4f2da3-b977-487d-8e69-d4258d31629d None None] MFA Rules not processed for user `356d396ae5cf47c3bc61f24f9e6dfa02`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:26:21.397862 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8e4f2da3-b977-487d-8e69-d4258d31629d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:26:21.460028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 688/1375] 10.222.0.52 () {58 vars in 1060 bytes} [Wed Jun 29 12:26:21 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:26:21.468399 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783462b-802d-42fe-a37f-4e92a32bd59b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:26:21.481598 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783462b-802d-42fe-a37f-4e92a32bd59b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:26:21.485294 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ea45b-8b93-47f9-b65b-79713a193e54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:26:21.486170 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ea45b-8b93-47f9-b65b-79713a193e54 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:21.486490 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ea45b-8b93-47f9-b65b-79713a193e54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:21.486775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ea45b-8b93-47f9-b65b-79713a193e54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:21.496353 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ea45b-8b93-47f9-b65b-79713a193e54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:26:21.497786 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ea45b-8b93-47f9-b65b-79713a193e54 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:26:21.512364 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 688/1376] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:26:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:26:22.409304 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece14529-35e1-45a0-896e-917703dfe334 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:22.409811 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece14529-35e1-45a0-896e-917703dfe334 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:22.409907 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece14529-35e1-45a0-896e-917703dfe334 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:22.411239 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 689/1377] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:26:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:26:22.416332 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a38b5c-c7a1-4686-ad90-3f1b7a82d11d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:22.416748 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a38b5c-c7a1-4686-ad90-3f1b7a82d11d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:22.416899 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a38b5c-c7a1-4686-ad90-3f1b7a82d11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:22.434522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-16a38b5c-c7a1-4686-ad90-3f1b7a82d11d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:26:22.435708 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-16a38b5c-c7a1-4686-ad90-3f1b7a82d11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:26:22.443635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 689/1378] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:26:22.488425 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece14529-35e1-45a0-896e-917703dfe334 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:26:22.502841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece14529-35e1-45a0-896e-917703dfe334 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:26:22.508227 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bfc90b-eff7-4eab-96b1-dbec474230ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:26:22.508922 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bfc90b-eff7-4eab-96b1-dbec474230ef service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:22.509237 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bfc90b-eff7-4eab-96b1-dbec474230ef service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:22.509498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bfc90b-eff7-4eab-96b1-dbec474230ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:22.518205 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bfc90b-eff7-4eab-96b1-dbec474230ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:26:22.519857 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bfc90b-eff7-4eab-96b1-dbec474230ef service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:26:22.524069 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:26:22.530965 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 690/1379] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:26:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:26:24.723184 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6b667-84df-4503-b2d2-81d80dd5ab12 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:24.723665 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6b667-84df-4503-b2d2-81d80dd5ab12 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:24.723665 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f6b667-84df-4503-b2d2-81d80dd5ab12 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:24.725083 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 690/1380] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:26:24.730664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008fe84-2ffd-495a-a1ec-732aa545b6b6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:24.731084 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008fe84-2ffd-495a-a1ec-732aa545b6b6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:24.731281 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008fe84-2ffd-495a-a1ec-732aa545b6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:24.750038 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5008fe84-2ffd-495a-a1ec-732aa545b6b6 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:26:24.751868 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5008fe84-2ffd-495a-a1ec-732aa545b6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:26:24.759853 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 691/1381] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:26:24.804650 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6b667-84df-4503-b2d2-81d80dd5ab12 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:26:24.819196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f6b667-84df-4503-b2d2-81d80dd5ab12 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:26:24.824876 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da6bebd-cd60-4a3f-b534-8e6ba292e8e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:26:24.825520 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6bebd-cd60-4a3f-b534-8e6ba292e8e4 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:24.825808 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6bebd-cd60-4a3f-b534-8e6ba292e8e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:24.826090 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6bebd-cd60-4a3f-b534-8e6ba292e8e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:24.834265 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da6bebd-cd60-4a3f-b534-8e6ba292e8e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:26:24.835872 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da6bebd-cd60-4a3f-b534-8e6ba292e8e4 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:26:24.845229 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 691/1382] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:26:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:26:26.973271 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfaf2ba-e337-4760-9508-ad4ffce0236d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:26.973789 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfaf2ba-e337-4760-9508-ad4ffce0236d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:26.973789 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfaf2ba-e337-4760-9508-ad4ffce0236d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:26.975378 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 692/1383] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:26:26.980461 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ae0cc-e3ec-4a5a-961a-2e81e8b9d6d9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:26.980899 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ae0cc-e3ec-4a5a-961a-2e81e8b9d6d9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:26.981051 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044ae0cc-e3ec-4a5a-961a-2e81e8b9d6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:26.999284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-044ae0cc-e3ec-4a5a-961a-2e81e8b9d6d9 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:26:27.000464 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-044ae0cc-e3ec-4a5a-961a-2e81e8b9d6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:26:27.008761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 692/1384] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:26:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:26:27.052169 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfaf2ba-e337-4760-9508-ad4ffce0236d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:26:27.069623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfaf2ba-e337-4760-9508-ad4ffce0236d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:26:27.075327 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8f95da-07dd-4acb-8157-aead185ac799 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:26:27.076174 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f95da-07dd-4acb-8157-aead185ac799 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:27.076624 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f95da-07dd-4acb-8157-aead185ac799 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:27.077045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8f95da-07dd-4acb-8157-aead185ac799 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:27.087812 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f95da-07dd-4acb-8157-aead185ac799 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:26:27.089732 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8f95da-07dd-4acb-8157-aead185ac799 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:26:27.102272 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 693/1385] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:26:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:26:29.229252 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5bb2e-5b37-4f0f-86f1-4cc9e184a959 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:29.229777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5bb2e-5b37-4f0f-86f1-4cc9e184a959 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:29.229885 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5bb2e-5b37-4f0f-86f1-4cc9e184a959 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:29.231459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 693/1386] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:26:29.237134 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874557a-9786-4537-b972-2ba7c70a7bc6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:29.237769 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874557a-9786-4537-b972-2ba7c70a7bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:29.237769 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874557a-9786-4537-b972-2ba7c70a7bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:29.256700 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5874557a-9786-4537-b972-2ba7c70a7bc6 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:26:29.257901 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5874557a-9786-4537-b972-2ba7c70a7bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:26:29.267738 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 694/1387] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:26:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:26:29.314638 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5bb2e-5b37-4f0f-86f1-4cc9e184a959 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:26:29.329691 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5bb2e-5b37-4f0f-86f1-4cc9e184a959 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:26:29.335504 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2edc67-dd07-4f72-8364-e720cdc9098c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:26:29.336127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2edc67-dd07-4f72-8364-e720cdc9098c service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:29.336415 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2edc67-dd07-4f72-8364-e720cdc9098c service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:29.336716 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2edc67-dd07-4f72-8364-e720cdc9098c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:29.349747 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2edc67-dd07-4f72-8364-e720cdc9098c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:26:29.351421 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2edc67-dd07-4f72-8364-e720cdc9098c service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:26:29.362331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 694/1388] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:26:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 12:26:31.929386 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed2dee-06a5-4abf-9021-527702b161b0 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:31.930303 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed2dee-06a5-4abf-9021-527702b161b0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:31.930303 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed2dee-06a5-4abf-9021-527702b161b0 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:31.931948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 695/1389] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:26:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:26:31.939723 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf633c-d222-444a-97e7-8c3897329c7c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:31.940352 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf633c-d222-444a-97e7-8c3897329c7c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:31.940352 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf633c-d222-444a-97e7-8c3897329c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:31.959681 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-03cf633c-d222-444a-97e7-8c3897329c7c None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:26:31.960971 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-03cf633c-d222-444a-97e7-8c3897329c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:26:31.970650 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 695/1390] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:26:32.026086 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed2dee-06a5-4abf-9021-527702b161b0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:26:32.043662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed2dee-06a5-4abf-9021-527702b161b0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:26:32.050159 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443caa4e-7041-4985-b762-c20ec547e445 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:26:32.051048 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443caa4e-7041-4985-b762-c20ec547e445 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:32.051400 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443caa4e-7041-4985-b762-c20ec547e445 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:32.051743 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443caa4e-7041-4985-b762-c20ec547e445 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:32.062015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443caa4e-7041-4985-b762-c20ec547e445 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:26:32.064634 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443caa4e-7041-4985-b762-c20ec547e445 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:26:32.076892 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 696/1391] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:26:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:26:34.177286 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035800c-96da-41b9-8afd-509ba1cf9000 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:34.177895 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035800c-96da-41b9-8afd-509ba1cf9000 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:34.177997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3035800c-96da-41b9-8afd-509ba1cf9000 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:34.180702 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 696/1392] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:26:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:26:34.186054 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e070cf-72fb-4776-ad13-4b1f2639fe3d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:34.186507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e070cf-72fb-4776-ad13-4b1f2639fe3d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:34.186639 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e070cf-72fb-4776-ad13-4b1f2639fe3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:34.205650 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e1e070cf-72fb-4776-ad13-4b1f2639fe3d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:26:34.206883 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e1e070cf-72fb-4776-ad13-4b1f2639fe3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:26:34.214631 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 697/1393] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:26:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:26:34.268731 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3035800c-96da-41b9-8afd-509ba1cf9000 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:26:34.287302 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3035800c-96da-41b9-8afd-509ba1cf9000 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:26:34.294021 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5388f3-2994-4466-ad1d-9beb33004581 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:26:34.294909 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5388f3-2994-4466-ad1d-9beb33004581 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:34.295264 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5388f3-2994-4466-ad1d-9beb33004581 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:34.295614 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5388f3-2994-4466-ad1d-9beb33004581 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:34.306464 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5388f3-2994-4466-ad1d-9beb33004581 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:26:34.308487 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5388f3-2994-4466-ad1d-9beb33004581 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:26:34.321497 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 697/1394] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:26:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 12:26:36.400113 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911fad1-4f70-4b6f-b674-b234aa3fff36 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:36.400639 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911fad1-4f70-4b6f-b674-b234aa3fff36 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:36.400767 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911fad1-4f70-4b6f-b674-b234aa3fff36 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:36.402171 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 698/1395] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:26:36.408005 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a900b6-ec6f-4b9b-b5c4-f0406419ceea None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:36.408428 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a900b6-ec6f-4b9b-b5c4-f0406419ceea None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:36.408632 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a900b6-ec6f-4b9b-b5c4-f0406419ceea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:36.427730 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-42a900b6-ec6f-4b9b-b5c4-f0406419ceea None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:26:36.428920 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-42a900b6-ec6f-4b9b-b5c4-f0406419ceea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:26:36.436646 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 698/1396] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:26:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:26:36.480627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911fad1-4f70-4b6f-b674-b234aa3fff36 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:26:36.494951 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911fad1-4f70-4b6f-b674-b234aa3fff36 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:26:36.500205 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbec774-1d10-4fb8-af17-cb068b6c133c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:26:36.501543 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbec774-1d10-4fb8-af17-cb068b6c133c service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:36.501821 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbec774-1d10-4fb8-af17-cb068b6c133c service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:36.502098 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbec774-1d10-4fb8-af17-cb068b6c133c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:36.510799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbec774-1d10-4fb8-af17-cb068b6c133c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:26:36.512390 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbec774-1d10-4fb8-af17-cb068b6c133c service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:26:36.522551 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 699/1397] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:26:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:26:36.814754 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2759787-82bc-4383-acca-1107ec36a5ea None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:36.815034 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2759787-82bc-4383-acca-1107ec36a5ea None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:36.815305 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2759787-82bc-4383-acca-1107ec36a5ea None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:36.816681 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 699/1398] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:26:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:26:36.821817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b9bcb-f6c8-4b49-a72e-998740a59994 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:36.822198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b9bcb-f6c8-4b49-a72e-998740a59994 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:36.822682 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b9bcb-f6c8-4b49-a72e-998740a59994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:36.841305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-317b9bcb-f6c8-4b49-a72e-998740a59994 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:26:36.842545 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-317b9bcb-f6c8-4b49-a72e-998740a59994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:26:36.850739 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 700/1399] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:26:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:26:36.908495 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b264c9-a691-49cd-acc5-5c89e04d75a9 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:36.908859 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b264c9-a691-49cd-acc5-5c89e04d75a9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:36.909206 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b264c9-a691-49cd-acc5-5c89e04d75a9 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:36.910676 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 700/1400] 10.222.0.52 () {54 vars in 962 bytes} [Wed Jun 29 12:26:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:26:36.915967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d26846-5f56-4bc0-a133-baa3b02fb58a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:36.916258 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d26846-5f56-4bc0-a133-baa3b02fb58a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:36.916560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d26846-5f56-4bc0-a133-baa3b02fb58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:36.965224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f7d26846-5f56-4bc0-a133-baa3b02fb58a None None] MFA Rules not processed for user `096234e9f15541c0b187dc8dd527b0fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:26:36.966370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f7d26846-5f56-4bc0-a133-baa3b02fb58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:26:37.026847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 701/1401] 10.222.0.52 () {58 vars in 1059 bytes} [Wed Jun 29 12:26:36 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:26:37.034882 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b264c9-a691-49cd-acc5-5c89e04d75a9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:26:37.045140 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b264c9-a691-49cd-acc5-5c89e04d75a9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:26:37.049401 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034d78a-8309-4b7e-996b-ecc508e74774 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '096234e9f15541c0b187dc8dd527b0fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:26:37.050032 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034d78a-8309-4b7e-996b-ecc508e74774 service glance] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:37.050315 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034d78a-8309-4b7e-996b-ecc508e74774 service glance] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:37.050618 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034d78a-8309-4b7e-996b-ecc508e74774 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:37.059490 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034d78a-8309-4b7e-996b-ecc508e74774 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:26:37.060913 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034d78a-8309-4b7e-996b-ecc508e74774 service glance] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:26:37.070651 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 701/1402] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:26:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:26:37.793983 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dde3cf-611a-4a83-8660-673b69598cea None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:37.794489 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dde3cf-611a-4a83-8660-673b69598cea None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:37.794598 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dde3cf-611a-4a83-8660-673b69598cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:37.842795 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f5dde3cf-611a-4a83-8660-673b69598cea None None] MFA Rules not processed for user `8c10852a728844c6ae0a48e3772d95b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:26:37.844005 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f5dde3cf-611a-4a83-8660-673b69598cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:26:37.884666 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:26:37.909632 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 702/1403] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jun 29 12:26:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:26:37.951279 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d50ef-736b-4bf9-bf39-4d3b4ea9fc9c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:37.951750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d50ef-736b-4bf9-bf39-4d3b4ea9fc9c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:37.951978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d50ef-736b-4bf9-bf39-4d3b4ea9fc9c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:37.953432 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 702/1404] 10.222.0.52 () {54 vars in 942 bytes} [Wed Jun 29 12:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:26:37.960029 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca583c7-f2b4-4abc-a6fc-f632dabce2dc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:37.960395 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca583c7-f2b4-4abc-a6fc-f632dabce2dc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:37.960746 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca583c7-f2b4-4abc-a6fc-f632dabce2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:38.009591 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-eca583c7-f2b4-4abc-a6fc-f632dabce2dc None None] MFA Rules not processed for user `209b0df45f76404da880a9ae0c637d67`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:26:38.010832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-eca583c7-f2b4-4abc-a6fc-f632dabce2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:26:38.083090 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 703/1405] 10.222.0.52 () {58 vars in 1039 bytes} [Wed Jun 29 12:26:37 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:26:38.092188 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36d50ef-736b-4bf9-bf39-4d3b4ea9fc9c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:26:38.107342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36d50ef-736b-4bf9-bf39-4d3b4ea9fc9c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:26:38.113022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1559ba1-a0b5-458f-a96c-71a3341f6541 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209b0df45f76404da880a9ae0c637d67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:26:38.113741 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1559ba1-a0b5-458f-a96c-71a3341f6541 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:38.114095 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1559ba1-a0b5-458f-a96c-71a3341f6541 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:38.114475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1559ba1-a0b5-458f-a96c-71a3341f6541 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:38.119597 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=63 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:26:38.123719 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1559ba1-a0b5-458f-a96c-71a3341f6541 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:26:38.126230 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1559ba1-a0b5-458f-a96c-71a3341f6541 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:26:38.134574 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 703/1406] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 12:26:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 12:26:38.593277 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f126ca9-2d5c-412d-947a-783572d68339 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:38.593561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f126ca9-2d5c-412d-947a-783572d68339 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:38.593843 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f126ca9-2d5c-412d-947a-783572d68339 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:38.598806 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 704/1407] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:26:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:26:38.601110 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2108d-193d-4c0b-91c9-c865d6051b60 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:38.601399 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2108d-193d-4c0b-91c9-c865d6051b60 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:38.601674 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2108d-193d-4c0b-91c9-c865d6051b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:38.620794 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-05a2108d-193d-4c0b-91c9-c865d6051b60 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:26:38.621908 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-05a2108d-193d-4c0b-91c9-c865d6051b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:26:38.632495 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 704/1408] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:26:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:26:38.678391 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f126ca9-2d5c-412d-947a-783572d68339 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:26:38.695313 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f126ca9-2d5c-412d-947a-783572d68339 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:26:38.700714 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdff0cf-aa68-4104-91c9-b12b3fed4e43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:26:38.702002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdff0cf-aa68-4104-91c9-b12b3fed4e43 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:26:38.702294 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdff0cf-aa68-4104-91c9-b12b3fed4e43 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:26:38.702613 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdff0cf-aa68-4104-91c9-b12b3fed4e43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:26:38.713601 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdff0cf-aa68-4104-91c9-b12b3fed4e43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:26:38.715549 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdff0cf-aa68-4104-91c9-b12b3fed4e43 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:26:38.727146 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 705/1409] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:26:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:26:48.614358 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:26:48.721999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=59 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:27:00.604621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a7ab9-ca46-4fc6-9823-8abf359af28b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:00.605202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a7ab9-ca46-4fc6-9823-8abf359af28b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:00.605202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a7ab9-ca46-4fc6-9823-8abf359af28b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:00.606521 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 705/1410] 10.222.0.52 () {54 vars in 918 bytes} [Wed Jun 29 12:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:27:00.611676 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711aa49-9bbe-4d08-b447-58333c9cd916 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:00.612124 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711aa49-9bbe-4d08-b447-58333c9cd916 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:00.612232 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4711aa49-9bbe-4d08-b447-58333c9cd916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:00.630654 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4711aa49-9bbe-4d08-b447-58333c9cd916 None None] MFA Rules not processed for user `c98135cacb9d4f22846ad33500b9bd1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:27:00.631870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4711aa49-9bbe-4d08-b447-58333c9cd916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:27:00.640285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 706/1411] 10.222.0.52 () {58 vars in 1015 bytes} [Wed Jun 29 12:27:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:27:00.654669 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd49be6-5ab7-44f7-941d-514d409b44a3 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:00.654955 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd49be6-5ab7-44f7-941d-514d409b44a3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:00.655232 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd49be6-5ab7-44f7-941d-514d409b44a3 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:00.656611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 706/1412] 10.222.0.52 () {54 vars in 918 bytes} [Wed Jun 29 12:27:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:27:00.661033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88197166-3b59-4e44-90c6-a1fa618a1c1e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:00.661329 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88197166-3b59-4e44-90c6-a1fa618a1c1e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:00.661614 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88197166-3b59-4e44-90c6-a1fa618a1c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:00.679968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-88197166-3b59-4e44-90c6-a1fa618a1c1e None None] MFA Rules not processed for user `3ac7391a3c41499d94e5f97478797313`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:27:00.681147 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-88197166-3b59-4e44-90c6-a1fa618a1c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:27:00.690481 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 707/1413] 10.222.0.52 () {58 vars in 1015 bytes} [Wed Jun 29 12:27:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:27:03.212092 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8a6fe-d096-4b33-95e0-3b11c228245c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:03.212624 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8a6fe-d096-4b33-95e0-3b11c228245c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:03.212624 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8a6fe-d096-4b33-95e0-3b11c228245c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:03.214014 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 12:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:27:03.219336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa70cf6-0ede-407e-b500-ceb5b414d062 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:03.219717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa70cf6-0ede-407e-b500-ceb5b414d062 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:03.219903 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa70cf6-0ede-407e-b500-ceb5b414d062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:03.238882 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3fa70cf6-0ede-407e-b500-ceb5b414d062 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:27:03.240086 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3fa70cf6-0ede-407e-b500-ceb5b414d062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:27:03.247862 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 12:27:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:27:03.291553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8a6fe-d096-4b33-95e0-3b11c228245c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:27:03.306166 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8a6fe-d096-4b33-95e0-3b11c228245c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:27:03.311507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44ca14-aaa3-4f7b-818f-487598cf003e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:27:03.312323 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44ca14-aaa3-4f7b-818f-487598cf003e service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:03.312741 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44ca14-aaa3-4f7b-818f-487598cf003e service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:03.313050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44ca14-aaa3-4f7b-818f-487598cf003e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:03.321373 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e44ca14-aaa3-4f7b-818f-487598cf003e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:27:03.322987 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e44ca14-aaa3-4f7b-818f-487598cf003e service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:27:03.332237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 708/1416] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:27:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:27:04.224921 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43533d1d-0de9-4e0f-b6e5-c6f5da92a902 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:04.225485 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43533d1d-0de9-4e0f-b6e5-c6f5da92a902 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:04.225485 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43533d1d-0de9-4e0f-b6e5-c6f5da92a902 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:04.226972 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 12:27:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:27:04.231883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0e549-95f2-4f41-86b1-665b92492108 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:04.232309 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0e549-95f2-4f41-86b1-665b92492108 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:04.232461 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0e549-95f2-4f41-86b1-665b92492108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:04.250790 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2ee0e549-95f2-4f41-86b1-665b92492108 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:27:04.251986 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2ee0e549-95f2-4f41-86b1-665b92492108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:27:04.259643 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 12:27:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:27:04.414498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43533d1d-0de9-4e0f-b6e5-c6f5da92a902 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:27:04.428343 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43533d1d-0de9-4e0f-b6e5-c6f5da92a902 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:27:04.433712 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb472deb-a694-4848-a2ff-cca670b24de6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:27:04.434339 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb472deb-a694-4848-a2ff-cca670b24de6 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:04.434658 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb472deb-a694-4848-a2ff-cca670b24de6 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:04.434936 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb472deb-a694-4848-a2ff-cca670b24de6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:04.442890 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb472deb-a694-4848-a2ff-cca670b24de6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:27:04.444452 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb472deb-a694-4848-a2ff-cca670b24de6 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:27:04.454163 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 710/1419] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:27:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:27:05.333498 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd46b51-c5f9-4fde-80f6-612871065ceb None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:05.334064 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd46b51-c5f9-4fde-80f6-612871065ceb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:05.334064 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd46b51-c5f9-4fde-80f6-612871065ceb None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:05.335493 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 12:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:27:05.340482 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6919e-f790-468a-b2f8-f64033297f37 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:05.340914 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6919e-f790-468a-b2f8-f64033297f37 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:05.341023 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6919e-f790-468a-b2f8-f64033297f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:05.361028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-44c6919e-f790-468a-b2f8-f64033297f37 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:27:05.362255 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-44c6919e-f790-468a-b2f8-f64033297f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:27:05.369966 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 12:27:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:27:05.526136 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd46b51-c5f9-4fde-80f6-612871065ceb None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:27:05.541122 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd46b51-c5f9-4fde-80f6-612871065ceb None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:27:05.546413 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33e871-a18e-4b41-95be-bf8a3ecc40aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:27:05.547074 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33e871-a18e-4b41-95be-bf8a3ecc40aa service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:05.547357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33e871-a18e-4b41-95be-bf8a3ecc40aa service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:05.547632 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33e871-a18e-4b41-95be-bf8a3ecc40aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:05.555628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c33e871-a18e-4b41-95be-bf8a3ecc40aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:27:05.557205 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c33e871-a18e-4b41-95be-bf8a3ecc40aa service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:27:05.566919 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 711/1422] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:27:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:27:06.422453 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696b348-7be5-4b53-bdb1-523103cf0bf7 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:06.422952 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696b348-7be5-4b53-bdb1-523103cf0bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:06.423070 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696b348-7be5-4b53-bdb1-523103cf0bf7 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:06.424439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 12:27:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:27:06.429361 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bab2bf-6702-4772-8ee3-95bc9b3abf0d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:06.429777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bab2bf-6702-4772-8ee3-95bc9b3abf0d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:06.429914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bab2bf-6702-4772-8ee3-95bc9b3abf0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:06.447282 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-90bab2bf-6702-4772-8ee3-95bc9b3abf0d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:27:06.448473 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-90bab2bf-6702-4772-8ee3-95bc9b3abf0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:27:06.456170 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 12:27:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:27:06.605214 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696b348-7be5-4b53-bdb1-523103cf0bf7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:27:06.620077 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696b348-7be5-4b53-bdb1-523103cf0bf7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:27:06.625467 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c23366-8da5-47f6-8547-6d1b182d6b6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:27:06.626098 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c23366-8da5-47f6-8547-6d1b182d6b6e service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:06.626386 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c23366-8da5-47f6-8547-6d1b182d6b6e service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:06.626699 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c23366-8da5-47f6-8547-6d1b182d6b6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:06.634781 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c23366-8da5-47f6-8547-6d1b182d6b6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:27:06.636349 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c23366-8da5-47f6-8547-6d1b182d6b6e service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:27:06.646147 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 713/1425] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:27:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:27:07.501667 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262db61-f433-4cf0-8b22-e8619f8b339b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:07.502197 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262db61-f433-4cf0-8b22-e8619f8b339b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:07.502311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262db61-f433-4cf0-8b22-e8619f8b339b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:07.503654 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 12:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:27:07.508408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2194d2-2e1e-4e0d-9cec-46c1bba0352d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:07.508959 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2194d2-2e1e-4e0d-9cec-46c1bba0352d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:07.508959 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2194d2-2e1e-4e0d-9cec-46c1bba0352d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:07.527595 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9a2194d2-2e1e-4e0d-9cec-46c1bba0352d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:27:07.528790 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9a2194d2-2e1e-4e0d-9cec-46c1bba0352d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:27:07.537172 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 12:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:27:07.691654 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262db61-f433-4cf0-8b22-e8619f8b339b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:27:07.705954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262db61-f433-4cf0-8b22-e8619f8b339b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:27:07.711185 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46637887-28a9-4adb-95f5-a22b9de5c2e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:27:07.711809 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46637887-28a9-4adb-95f5-a22b9de5c2e3 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:07.712091 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46637887-28a9-4adb-95f5-a22b9de5c2e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:07.712366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46637887-28a9-4adb-95f5-a22b9de5c2e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:07.720884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46637887-28a9-4adb-95f5-a22b9de5c2e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:27:07.722469 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46637887-28a9-4adb-95f5-a22b9de5c2e3 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:27:07.731705 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 714/1428] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:27:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:27:08.545945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b0e86-e5d3-47ae-9d07-7aa547a36e43 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:08.546598 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b0e86-e5d3-47ae-9d07-7aa547a36e43 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:08.546753 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0b0e86-e5d3-47ae-9d07-7aa547a36e43 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:08.548121 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 12:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:27:08.553156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23834db-4da3-4a66-8c31-a36947f2cb9d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:08.553562 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23834db-4da3-4a66-8c31-a36947f2cb9d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:08.553715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23834db-4da3-4a66-8c31-a36947f2cb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:08.571765 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b23834db-4da3-4a66-8c31-a36947f2cb9d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:27:08.572929 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b23834db-4da3-4a66-8c31-a36947f2cb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:27:08.581174 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 12:27:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:27:08.731231 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b0e86-e5d3-47ae-9d07-7aa547a36e43 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:27:08.746097 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0b0e86-e5d3-47ae-9d07-7aa547a36e43 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:27:08.751600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d443a80-d535-4d9b-9b72-45e69b00a17a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:27:08.752234 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d443a80-d535-4d9b-9b72-45e69b00a17a service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:08.752520 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d443a80-d535-4d9b-9b72-45e69b00a17a service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:08.752798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d443a80-d535-4d9b-9b72-45e69b00a17a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:08.760743 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d443a80-d535-4d9b-9b72-45e69b00a17a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:27:08.762313 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d443a80-d535-4d9b-9b72-45e69b00a17a service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:27:08.771613 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 716/1431] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:27:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:27:09.570000 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd61a50-fc60-43e7-8c2a-11390a593a31 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:09.570588 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd61a50-fc60-43e7-8c2a-11390a593a31 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:09.570690 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd61a50-fc60-43e7-8c2a-11390a593a31 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:09.572034 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 12:27:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:27:09.577112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f345a40-0d74-4efe-9d4b-07b422d3ffb9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:09.577719 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f345a40-0d74-4efe-9d4b-07b422d3ffb9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:09.578325 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f345a40-0d74-4efe-9d4b-07b422d3ffb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:09.597054 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6f345a40-0d74-4efe-9d4b-07b422d3ffb9 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:27:09.598239 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6f345a40-0d74-4efe-9d4b-07b422d3ffb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:27:09.605905 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 12:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:27:09.760063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd61a50-fc60-43e7-8c2a-11390a593a31 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:27:09.774791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd61a50-fc60-43e7-8c2a-11390a593a31 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:27:09.779980 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2418d2b-e65f-4e21-ba69-59798224ccf9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:27:09.781158 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2418d2b-e65f-4e21-ba69-59798224ccf9 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:09.781438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2418d2b-e65f-4e21-ba69-59798224ccf9 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:09.781713 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2418d2b-e65f-4e21-ba69-59798224ccf9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:09.789541 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2418d2b-e65f-4e21-ba69-59798224ccf9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:27:09.791144 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2418d2b-e65f-4e21-ba69-59798224ccf9 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:27:09.800380 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 717/1434] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:27:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:27:10.611388 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e3d60-c6f0-445a-aa26-1bfe1d3021b1 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:10.611990 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e3d60-c6f0-445a-aa26-1bfe1d3021b1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:10.611990 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e3d60-c6f0-445a-aa26-1bfe1d3021b1 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:10.613325 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 12:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:27:10.618279 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d4c28-4570-43b8-aec0-3403ffc8d7ac None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:10.618736 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d4c28-4570-43b8-aec0-3403ffc8d7ac None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:10.618924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d4c28-4570-43b8-aec0-3403ffc8d7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:10.637256 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e06d4c28-4570-43b8-aec0-3403ffc8d7ac None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:27:10.638475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e06d4c28-4570-43b8-aec0-3403ffc8d7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:27:10.646256 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 12:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:27:10.797548 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e3d60-c6f0-445a-aa26-1bfe1d3021b1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:27:10.811888 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e3d60-c6f0-445a-aa26-1bfe1d3021b1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:27:10.817283 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11212ca4-5fa5-4637-8e5a-869fcbc5ca72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:27:10.817921 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11212ca4-5fa5-4637-8e5a-869fcbc5ca72 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:10.818234 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11212ca4-5fa5-4637-8e5a-869fcbc5ca72 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:10.818555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11212ca4-5fa5-4637-8e5a-869fcbc5ca72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:10.826599 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11212ca4-5fa5-4637-8e5a-869fcbc5ca72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:27:10.828178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11212ca4-5fa5-4637-8e5a-869fcbc5ca72 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:27:10.837477 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 719/1437] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:27:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:27:11.636644 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ce0fa-0b3e-4c3e-a8d6-7942e8e1761c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:11.637165 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ce0fa-0b3e-4c3e-a8d6-7942e8e1761c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:11.637165 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ce0fa-0b3e-4c3e-a8d6-7942e8e1761c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:11.638499 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 12:27:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:27:11.643888 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1867271-1f63-4e2f-9729-fd82f83c526c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:11.644301 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1867271-1f63-4e2f-9729-fd82f83c526c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:11.644450 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1867271-1f63-4e2f-9729-fd82f83c526c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:11.662103 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a1867271-1f63-4e2f-9729-fd82f83c526c None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:27:11.663309 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a1867271-1f63-4e2f-9729-fd82f83c526c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:27:11.670951 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 12:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:27:11.819444 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ce0fa-0b3e-4c3e-a8d6-7942e8e1761c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:27:11.833994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ce0fa-0b3e-4c3e-a8d6-7942e8e1761c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:27:11.839204 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b438271-71a2-43d9-8589-7bf3dfcf1a6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:27:11.839830 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b438271-71a2-43d9-8589-7bf3dfcf1a6e service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:11.840112 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b438271-71a2-43d9-8589-7bf3dfcf1a6e service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:11.840388 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b438271-71a2-43d9-8589-7bf3dfcf1a6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:11.848336 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b438271-71a2-43d9-8589-7bf3dfcf1a6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:27:11.849902 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b438271-71a2-43d9-8589-7bf3dfcf1a6e service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:27:11.859016 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 720/1440] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:27:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:27:13.822333 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d32eb6-5d91-4c27-b4d7-b7ebf64965e8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:13.822917 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d32eb6-5d91-4c27-b4d7-b7ebf64965e8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:13.822917 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d32eb6-5d91-4c27-b4d7-b7ebf64965e8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:13.824278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 12:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:27:13.830102 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ee4ce-284d-4f0b-ac21-3001027ba0fd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:13.830519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ee4ce-284d-4f0b-ac21-3001027ba0fd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:13.830707 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ee4ce-284d-4f0b-ac21-3001027ba0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:13.848155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-476ee4ce-284d-4f0b-ac21-3001027ba0fd None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:27:13.849347 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-476ee4ce-284d-4f0b-ac21-3001027ba0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:27:13.857023 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 12:27:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:27:13.895776 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d32eb6-5d91-4c27-b4d7-b7ebf64965e8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:27:13.906516 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d32eb6-5d91-4c27-b4d7-b7ebf64965e8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:27:13.909953 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b5733-608f-4cfe-93a0-428006891ac6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:27:13.910612 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b5733-608f-4cfe-93a0-428006891ac6 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:13.910895 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b5733-608f-4cfe-93a0-428006891ac6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:13.911173 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b5733-608f-4cfe-93a0-428006891ac6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:13.920270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874b5733-608f-4cfe-93a0-428006891ac6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:27:13.922801 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874b5733-608f-4cfe-93a0-428006891ac6 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:27:13.934900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 722/1443] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:27:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:27:16.164775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c46dff-e950-41eb-995a-686b070fa796 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:16.165333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c46dff-e950-41eb-995a-686b070fa796 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:16.165333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c46dff-e950-41eb-995a-686b070fa796 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:16.166764 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 12:27:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:27:16.172783 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a45c7d-4f57-4f0b-a8c8-6637a4ea1e8e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:16.173244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a45c7d-4f57-4f0b-a8c8-6637a4ea1e8e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:16.173351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a45c7d-4f57-4f0b-a8c8-6637a4ea1e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:16.190719 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d5a45c7d-4f57-4f0b-a8c8-6637a4ea1e8e None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:27:16.191919 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d5a45c7d-4f57-4f0b-a8c8-6637a4ea1e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:27:16.199598 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 12:27:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:27:16.237268 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c46dff-e950-41eb-995a-686b070fa796 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:27:16.247850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c46dff-e950-41eb-995a-686b070fa796 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:27:16.252114 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca342f9-e536-4ccf-a562-692f91118c88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:27:16.252752 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca342f9-e536-4ccf-a562-692f91118c88 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:16.253041 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca342f9-e536-4ccf-a562-692f91118c88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:16.253322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca342f9-e536-4ccf-a562-692f91118c88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:16.262130 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca342f9-e536-4ccf-a562-692f91118c88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:27:16.263600 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca342f9-e536-4ccf-a562-692f91118c88 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:27:16.273628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 723/1446] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:27:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:27:18.625722 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea5c94-ca5f-4125-9a6b-5c5c3f17be19 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:18.626258 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea5c94-ca5f-4125-9a6b-5c5c3f17be19 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:18.626258 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ea5c94-ca5f-4125-9a6b-5c5c3f17be19 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:18.627661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 12:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:27:18.632784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b8d12-f92c-44ff-9aa9-290f8aa477ba None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:18.633181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b8d12-f92c-44ff-9aa9-290f8aa477ba None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:18.633359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b8d12-f92c-44ff-9aa9-290f8aa477ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:18.656790 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a89b8d12-f92c-44ff-9aa9-290f8aa477ba None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:27:18.658896 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a89b8d12-f92c-44ff-9aa9-290f8aa477ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:27:18.666674 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 12:27:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:27:18.705309 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea5c94-ca5f-4125-9a6b-5c5c3f17be19 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:27:18.715940 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ea5c94-ca5f-4125-9a6b-5c5c3f17be19 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:27:18.720638 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e17333-2252-4cb9-ab80-b796f8b2bd9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:27:18.721285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e17333-2252-4cb9-ab80-b796f8b2bd9f service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:18.721568 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e17333-2252-4cb9-ab80-b796f8b2bd9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:18.721846 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e17333-2252-4cb9-ab80-b796f8b2bd9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:18.730740 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e17333-2252-4cb9-ab80-b796f8b2bd9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:27:18.732202 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e17333-2252-4cb9-ab80-b796f8b2bd9f service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:27:18.742110 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 725/1449] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:27:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:27:20.914550 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffab2d-ecce-45d0-a62d-6ac856ab03d1 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:20.915116 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffab2d-ecce-45d0-a62d-6ac856ab03d1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:20.915116 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffab2d-ecce-45d0-a62d-6ac856ab03d1 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:20.916496 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 12:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:27:20.922376 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29da02d-8a4b-46e6-bfc4-68beb54425f6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:20.922841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29da02d-8a4b-46e6-bfc4-68beb54425f6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:20.922975 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29da02d-8a4b-46e6-bfc4-68beb54425f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:20.941782 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b29da02d-8a4b-46e6-bfc4-68beb54425f6 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:27:20.943005 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b29da02d-8a4b-46e6-bfc4-68beb54425f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:27:20.950755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 12:27:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:27:20.992923 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffab2d-ecce-45d0-a62d-6ac856ab03d1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:27:21.004368 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffab2d-ecce-45d0-a62d-6ac856ab03d1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:27:21.007827 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b765a6-201b-4337-a76e-29a5e00fe792 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:27:21.008454 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b765a6-201b-4337-a76e-29a5e00fe792 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:21.008737 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b765a6-201b-4337-a76e-29a5e00fe792 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:21.009015 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b765a6-201b-4337-a76e-29a5e00fe792 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:21.017607 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b765a6-201b-4337-a76e-29a5e00fe792 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:27:21.019055 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b765a6-201b-4337-a76e-29a5e00fe792 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:27:21.029163 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 726/1452] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:27:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:27:23.257144 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c84f1a-7b5e-4cc7-a409-79647fe784d9 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:23.257757 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c84f1a-7b5e-4cc7-a409-79647fe784d9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:23.257757 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c84f1a-7b5e-4cc7-a409-79647fe784d9 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:23.259478 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 12:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:27:23.266636 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032f4ef-5321-447d-b8b4-a6802ccaac8e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:23.267151 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032f4ef-5321-447d-b8b4-a6802ccaac8e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:23.267250 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032f4ef-5321-447d-b8b4-a6802ccaac8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:23.286594 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b032f4ef-5321-447d-b8b4-a6802ccaac8e None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:27:23.287898 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b032f4ef-5321-447d-b8b4-a6802ccaac8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:27:23.296009 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 12:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:27:23.335515 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c84f1a-7b5e-4cc7-a409-79647fe784d9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:27:23.348362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c84f1a-7b5e-4cc7-a409-79647fe784d9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:27:23.353110 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddeaa11-8dae-4de0-8714-539795dfa6c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:27:23.354827 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddeaa11-8dae-4de0-8714-539795dfa6c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:23.355973 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddeaa11-8dae-4de0-8714-539795dfa6c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:23.356261 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddeaa11-8dae-4de0-8714-539795dfa6c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:23.367198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddeaa11-8dae-4de0-8714-539795dfa6c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:27:23.369695 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddeaa11-8dae-4de0-8714-539795dfa6c1 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:27:23.383846 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 728/1455] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:27:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:27:25.512566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a4343-753b-494e-b36f-4240d78cd7eb None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:25.513089 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a4343-753b-494e-b36f-4240d78cd7eb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:25.513186 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a4343-753b-494e-b36f-4240d78cd7eb None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:25.515169 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 12:27:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:27:25.520524 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d826ad-fa9e-4e8a-bd14-1035ea374295 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:25.520996 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d826ad-fa9e-4e8a-bd14-1035ea374295 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:25.521111 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d826ad-fa9e-4e8a-bd14-1035ea374295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:25.539534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f6d826ad-fa9e-4e8a-bd14-1035ea374295 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:27:25.540739 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f6d826ad-fa9e-4e8a-bd14-1035ea374295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:27:25.548670 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 12:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:27:25.587139 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a4343-753b-494e-b36f-4240d78cd7eb None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:27:25.598315 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a4343-753b-494e-b36f-4240d78cd7eb None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:27:25.601746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f836ca65-01a3-4b27-bad7-2c335a09cdd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:27:25.602371 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836ca65-01a3-4b27-bad7-2c335a09cdd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:25.602683 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836ca65-01a3-4b27-bad7-2c335a09cdd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:25.602961 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f836ca65-01a3-4b27-bad7-2c335a09cdd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:25.611633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836ca65-01a3-4b27-bad7-2c335a09cdd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:27:25.613053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f836ca65-01a3-4b27-bad7-2c335a09cdd8 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:27:25.622222 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 729/1458] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:27:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:27:27.704439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5f1c7-6cff-436a-8a7c-c757e3ff7af6 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:27.704994 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5f1c7-6cff-436a-8a7c-c757e3ff7af6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:27.704994 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5f1c7-6cff-436a-8a7c-c757e3ff7af6 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:27.706278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 12:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:27:27.711525 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c23e5-cc17-438e-89b4-29f308b77396 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:27.711905 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c23e5-cc17-438e-89b4-29f308b77396 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:27.712090 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60c23e5-cc17-438e-89b4-29f308b77396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:27.729342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e60c23e5-cc17-438e-89b4-29f308b77396 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:27:27.730573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e60c23e5-cc17-438e-89b4-29f308b77396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:27:27.738834 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 12:27:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:27:27.776595 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5f1c7-6cff-436a-8a7c-c757e3ff7af6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:27:27.787448 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5f1c7-6cff-436a-8a7c-c757e3ff7af6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:27:27.790913 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3e9e9-136a-4d72-bf1b-70691f367b5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:27:27.791570 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3e9e9-136a-4d72-bf1b-70691f367b5f service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:27.791854 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3e9e9-136a-4d72-bf1b-70691f367b5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:27.792129 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3e9e9-136a-4d72-bf1b-70691f367b5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:27.801618 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3e9e9-136a-4d72-bf1b-70691f367b5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:27:27.803083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3e9e9-136a-4d72-bf1b-70691f367b5f service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:27:27.812908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 731/1461] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:27:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:27:29.876621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92022683-7088-4aba-bd59-3bd1f47977dc None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:29.877187 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92022683-7088-4aba-bd59-3bd1f47977dc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:29.877187 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92022683-7088-4aba-bd59-3bd1f47977dc None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:29.878564 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 12:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:27:29.883894 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5fa2ae-0de9-442a-9e7c-b2c0723a4b76 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:29.884294 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5fa2ae-0de9-442a-9e7c-b2c0723a4b76 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:29.884468 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5fa2ae-0de9-442a-9e7c-b2c0723a4b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:29.891261 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=79 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:27:29.903803 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8c5fa2ae-0de9-442a-9e7c-b2c0723a4b76 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:27:29.905206 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8c5fa2ae-0de9-442a-9e7c-b2c0723a4b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:27:29.913046 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 12:27:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:27:29.950518 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92022683-7088-4aba-bd59-3bd1f47977dc None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:27:29.956772 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=73 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:27:29.963064 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92022683-7088-4aba-bd59-3bd1f47977dc None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:27:29.966570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5663f-70c3-4569-9abd-d45c5776782b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:27:29.967780 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5663f-70c3-4569-9abd-d45c5776782b service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:29.968073 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5663f-70c3-4569-9abd-d45c5776782b service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:29.968350 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5663f-70c3-4569-9abd-d45c5776782b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:29.977172 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a5663f-70c3-4569-9abd-d45c5776782b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:27:29.978630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a5663f-70c3-4569-9abd-d45c5776782b service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:27:29.988233 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 732/1464] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:27:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:27:32.070581 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092fcae-47e0-4e54-9dc7-d14cdf3a7587 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:32.071147 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092fcae-47e0-4e54-9dc7-d14cdf3a7587 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:32.071147 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092fcae-47e0-4e54-9dc7-d14cdf3a7587 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:32.072495 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 12:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:27:32.079020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375810c-cfc0-4485-bf84-e2bf3b74b152 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:32.079607 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375810c-cfc0-4485-bf84-e2bf3b74b152 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:32.079607 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375810c-cfc0-4485-bf84-e2bf3b74b152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:32.096788 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c375810c-cfc0-4485-bf84-e2bf3b74b152 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:27:32.097962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c375810c-cfc0-4485-bf84-e2bf3b74b152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:27:32.105584 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 12:27:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:27:32.142144 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092fcae-47e0-4e54-9dc7-d14cdf3a7587 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:27:32.153064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092fcae-47e0-4e54-9dc7-d14cdf3a7587 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:27:32.156927 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c37bce-5ec8-4050-8f33-9c978077d13e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:27:32.157561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c37bce-5ec8-4050-8f33-9c978077d13e service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:32.157849 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c37bce-5ec8-4050-8f33-9c978077d13e service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:32.158129 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c37bce-5ec8-4050-8f33-9c978077d13e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:32.167080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c37bce-5ec8-4050-8f33-9c978077d13e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:27:32.168505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c37bce-5ec8-4050-8f33-9c978077d13e service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:27:32.178702 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 734/1467] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:27:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:27:34.249339 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ecd526-046b-40f9-a75e-9d44720f4143 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:34.249929 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ecd526-046b-40f9-a75e-9d44720f4143 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:34.249929 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ecd526-046b-40f9-a75e-9d44720f4143 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:34.251338 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 12:27:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:27:34.256474 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2f859-3e4f-4a7b-a9cf-0b174fe61125 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:34.256878 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2f859-3e4f-4a7b-a9cf-0b174fe61125 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:34.257049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc2f859-3e4f-4a7b-a9cf-0b174fe61125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:34.275384 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dbc2f859-3e4f-4a7b-a9cf-0b174fe61125 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:27:34.276578 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dbc2f859-3e4f-4a7b-a9cf-0b174fe61125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:27:34.284666 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 12:27:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:27:34.321720 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ecd526-046b-40f9-a75e-9d44720f4143 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:27:34.332648 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ecd526-046b-40f9-a75e-9d44720f4143 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:27:34.336165 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb5da8-b7cb-4c01-8ac5-451e63336ecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:27:34.336801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb5da8-b7cb-4c01-8ac5-451e63336ecd service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:34.337092 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb5da8-b7cb-4c01-8ac5-451e63336ecd service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:34.337378 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb5da8-b7cb-4c01-8ac5-451e63336ecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:34.347003 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb5da8-b7cb-4c01-8ac5-451e63336ecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:27:34.348446 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb5da8-b7cb-4c01-8ac5-451e63336ecd service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:27:34.358645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 735/1470] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:27:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:27:44.270095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=11 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:27:44.353676 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=14 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:27:54.201455 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3055fbb-2aa2-40c0-b684-f9ddea5c61f3 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:54.202224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3055fbb-2aa2-40c0-b684-f9ddea5c61f3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:54.202224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3055fbb-2aa2-40c0-b684-f9ddea5c61f3 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:54.203419 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 736/1471] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:27:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:27:54.208910 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee45810-1d65-47cf-aabf-f8a81b25c775 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:54.209367 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee45810-1d65-47cf-aabf-f8a81b25c775 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:54.209501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee45810-1d65-47cf-aabf-f8a81b25c775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:54.226812 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bee45810-1d65-47cf-aabf-f8a81b25c775 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:27:54.227994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bee45810-1d65-47cf-aabf-f8a81b25c775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:27:54.236713 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 736/1472] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:27:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:27:54.283862 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecb349-ad00-4b88-af90-0aba2e24358e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:54.284158 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecb349-ad00-4b88-af90-0aba2e24358e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:54.284442 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecb349-ad00-4b88-af90-0aba2e24358e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:54.285814 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 737/1473] 10.222.0.52 () {54 vars in 962 bytes} [Wed Jun 29 12:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:27:54.290651 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def057b0-b994-42a3-8fe2-0f435f3188fe None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:54.290943 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def057b0-b994-42a3-8fe2-0f435f3188fe None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:54.291225 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def057b0-b994-42a3-8fe2-0f435f3188fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:54.309673 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-def057b0-b994-42a3-8fe2-0f435f3188fe None None] MFA Rules not processed for user `096234e9f15541c0b187dc8dd527b0fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:27:54.310930 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-def057b0-b994-42a3-8fe2-0f435f3188fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:27:54.317219 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 737/1474] 10.222.0.52 () {58 vars in 1059 bytes} [Wed Jun 29 12:27:54 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:27:54.324376 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ecb349-ad00-4b88-af90-0aba2e24358e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:27:54.335434 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ecb349-ad00-4b88-af90-0aba2e24358e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:27:54.338900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b630f4-a8c7-464a-a4d9-0d128fb6ad15 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '096234e9f15541c0b187dc8dd527b0fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:27:54.339532 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b630f4-a8c7-464a-a4d9-0d128fb6ad15 service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:54.339820 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b630f4-a8c7-464a-a4d9-0d128fb6ad15 service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:54.340101 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b630f4-a8c7-464a-a4d9-0d128fb6ad15 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:54.349906 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b630f4-a8c7-464a-a4d9-0d128fb6ad15 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:27:54.351390 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b630f4-a8c7-464a-a4d9-0d128fb6ad15 service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:27:54.361217 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 738/1475] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:27:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:27:56.472863 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417114eb-9e80-48af-ae0b-34e4d8e3726a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:56.473819 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417114eb-9e80-48af-ae0b-34e4d8e3726a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:56.474113 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417114eb-9e80-48af-ae0b-34e4d8e3726a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:56.475670 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 738/1476] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:27:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:27:56.481126 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616599e-25bc-484c-8248-12c817126e74 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:56.481596 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616599e-25bc-484c-8248-12c817126e74 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:56.481855 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0616599e-25bc-484c-8248-12c817126e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:56.499178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0616599e-25bc-484c-8248-12c817126e74 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:27:56.500379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0616599e-25bc-484c-8248-12c817126e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:27:56.508104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 739/1477] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:27:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:27:56.532380 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417114eb-9e80-48af-ae0b-34e4d8e3726a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:27:56.544727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417114eb-9e80-48af-ae0b-34e4d8e3726a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:27:56.549126 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a71031-b8af-49a8-8ee7-11c0314b128f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:27:56.549763 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a71031-b8af-49a8-8ee7-11c0314b128f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:56.550052 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a71031-b8af-49a8-8ee7-11c0314b128f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:56.550332 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a71031-b8af-49a8-8ee7-11c0314b128f admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:56.551105 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a71031-b8af-49a8-8ee7-11c0314b128f admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:27:56.552855 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-44a71031-b8af-49a8-8ee7-11c0314b128f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 12:27:56.552855 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jun 29 12:27:56.552855 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: Jun 29 12:27:56.553990 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a71031-b8af-49a8-8ee7-11c0314b128f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:27:56.573317 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 739/1478] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jun 29 12:27:56 2022] GET /identity/v3/projects => generated 2553 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:27:58.643830 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65c96b-e8c2-4d27-95ee-14a308afb7bb None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:58.644354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65c96b-e8c2-4d27-95ee-14a308afb7bb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:58.644354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65c96b-e8c2-4d27-95ee-14a308afb7bb None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:58.645680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 740/1479] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:27:58.651244 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b627e1e-cb73-4c78-a0ed-b5c3796e85e0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:58.651702 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b627e1e-cb73-4c78-a0ed-b5c3796e85e0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:58.651816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b627e1e-cb73-4c78-a0ed-b5c3796e85e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:58.669679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2b627e1e-cb73-4c78-a0ed-b5c3796e85e0 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:27:58.670880 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2b627e1e-cb73-4c78-a0ed-b5c3796e85e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:27:58.678515 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 740/1480] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:27:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:27:58.720959 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65c96b-e8c2-4d27-95ee-14a308afb7bb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:27:58.735787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65c96b-e8c2-4d27-95ee-14a308afb7bb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:27:58.741065 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e1069-7f25-48a4-86d2-40e653ae9ba6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:27:58.741706 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e1069-7f25-48a4-86d2-40e653ae9ba6 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:27:58.741994 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e1069-7f25-48a4-86d2-40e653ae9ba6 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:27:58.742279 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e1069-7f25-48a4-86d2-40e653ae9ba6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:27:58.750947 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e1069-7f25-48a4-86d2-40e653ae9ba6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:27:58.753536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e1069-7f25-48a4-86d2-40e653ae9ba6 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:27:58.763322 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 741/1481] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:27:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:28:00.959812 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04592e98-c4a5-4468-8d2c-e858e664e100 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:28:00.960332 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04592e98-c4a5-4468-8d2c-e858e664e100 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:28:00.960459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04592e98-c4a5-4468-8d2c-e858e664e100 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:28:00.961743 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 741/1482] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:28:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:28:00.967403 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5915b37-f5b6-406f-96d1-8cfcdbd4868d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:28:00.967816 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5915b37-f5b6-406f-96d1-8cfcdbd4868d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:28:00.967961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5915b37-f5b6-406f-96d1-8cfcdbd4868d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:28:00.985379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d5915b37-f5b6-406f-96d1-8cfcdbd4868d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:28:00.986532 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d5915b37-f5b6-406f-96d1-8cfcdbd4868d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:28:00.994355 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 742/1483] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:28:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:28:01.030945 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04592e98-c4a5-4468-8d2c-e858e664e100 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:28:01.041544 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04592e98-c4a5-4468-8d2c-e858e664e100 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:28:01.045035 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf86017-ef8f-4b24-8dd5-b8126ce832f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:28:01.045653 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf86017-ef8f-4b24-8dd5-b8126ce832f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:28:01.045935 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf86017-ef8f-4b24-8dd5-b8126ce832f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:28:01.046214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf86017-ef8f-4b24-8dd5-b8126ce832f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:28:01.054882 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf86017-ef8f-4b24-8dd5-b8126ce832f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:28:01.056306 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf86017-ef8f-4b24-8dd5-b8126ce832f1 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:28:01.066085 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 742/1484] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:28:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:28:03.174239 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb7bb8-497a-4732-8592-59bd390ff62c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:28:03.174766 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb7bb8-497a-4732-8592-59bd390ff62c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:28:03.174885 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb7bb8-497a-4732-8592-59bd390ff62c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:28:03.176404 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 743/1485] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:28:03.181640 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78733ce8-ad3a-4cda-bb77-aa9e37d13ee4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:28:03.182062 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78733ce8-ad3a-4cda-bb77-aa9e37d13ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:28:03.182288 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78733ce8-ad3a-4cda-bb77-aa9e37d13ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:28:03.199736 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-78733ce8-ad3a-4cda-bb77-aa9e37d13ee4 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:28:03.201429 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-78733ce8-ad3a-4cda-bb77-aa9e37d13ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:28:03.209293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 743/1486] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:28:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:28:03.247375 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb7bb8-497a-4732-8592-59bd390ff62c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:28:03.258196 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb7bb8-497a-4732-8592-59bd390ff62c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:28:03.261724 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0e576-e33b-445d-a0aa-0fc019418f53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:28:03.262364 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0e576-e33b-445d-a0aa-0fc019418f53 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:28:03.263318 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0e576-e33b-445d-a0aa-0fc019418f53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:28:03.263593 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0e576-e33b-445d-a0aa-0fc019418f53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:28:03.272792 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0e576-e33b-445d-a0aa-0fc019418f53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:28:03.274199 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0e576-e33b-445d-a0aa-0fc019418f53 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:28:03.284523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 744/1487] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:28:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:28:05.540233 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172ef7e-a7dc-447c-be0f-58dfbe6dae26 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:28:05.540775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172ef7e-a7dc-447c-be0f-58dfbe6dae26 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:28:05.540775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c172ef7e-a7dc-447c-be0f-58dfbe6dae26 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:28:05.542186 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 744/1488] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:28:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:28:05.548157 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f67ca-d65b-40dd-87f1-1cfa3629f96c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:28:05.548566 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f67ca-d65b-40dd-87f1-1cfa3629f96c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:28:05.548721 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58f67ca-d65b-40dd-87f1-1cfa3629f96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:28:05.566916 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a58f67ca-d65b-40dd-87f1-1cfa3629f96c None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:28:05.568138 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a58f67ca-d65b-40dd-87f1-1cfa3629f96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:28:05.575927 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 745/1489] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:28:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:28:05.616126 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172ef7e-a7dc-447c-be0f-58dfbe6dae26 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:28:05.627675 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c172ef7e-a7dc-447c-be0f-58dfbe6dae26 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:28:05.633699 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee684c-0349-4280-b8ae-1214470a1d7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:28:05.634334 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee684c-0349-4280-b8ae-1214470a1d7a service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:28:05.634648 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee684c-0349-4280-b8ae-1214470a1d7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:28:05.634930 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee684c-0349-4280-b8ae-1214470a1d7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:28:05.645007 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee684c-0349-4280-b8ae-1214470a1d7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:28:05.646447 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee684c-0349-4280-b8ae-1214470a1d7a service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:28:05.656150 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 745/1490] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:28:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:28:07.856155 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5d2f3-c557-41ec-9fd4-9f168c9f2505 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:28:07.856694 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5d2f3-c557-41ec-9fd4-9f168c9f2505 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:28:07.856814 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5d2f3-c557-41ec-9fd4-9f168c9f2505 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:28:07.858184 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 746/1491] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jun 29 12:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:28:07.863416 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc883243-8b2a-4f16-8757-61db4ad381fd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:28:07.863814 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc883243-8b2a-4f16-8757-61db4ad381fd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:28:07.863980 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc883243-8b2a-4f16-8757-61db4ad381fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:28:07.881817 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-dc883243-8b2a-4f16-8757-61db4ad381fd None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:28:07.883027 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dc883243-8b2a-4f16-8757-61db4ad381fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:28:07.891289 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 746/1492] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jun 29 12:28:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:28:07.930481 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5d2f3-c557-41ec-9fd4-9f168c9f2505 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:28:07.941845 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5d2f3-c557-41ec-9fd4-9f168c9f2505 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:28:07.945407 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4748eb8-bec1-4d69-afe9-15cc9d64a404 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:28:07.946080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4748eb8-bec1-4d69-afe9-15cc9d64a404 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:28:07.946367 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4748eb8-bec1-4d69-afe9-15cc9d64a404 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:28:07.946699 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4748eb8-bec1-4d69-afe9-15cc9d64a404 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:28:07.956252 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4748eb8-bec1-4d69-afe9-15cc9d64a404 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:28:07.957733 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4748eb8-bec1-4d69-afe9-15cc9d64a404 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:28:07.967363 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 747/1493] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:28:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:28:17.876801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=33 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:28:17.963001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=33 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:29:34.431869 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ff691-6a28-460c-99e1-8eee48d43cdb None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:29:34.432406 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ff691-6a28-460c-99e1-8eee48d43cdb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:29:34.432504 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ff691-6a28-460c-99e1-8eee48d43cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:29:34.450609 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f87ff691-6a28-460c-99e1-8eee48d43cdb None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:29:34.451798 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f87ff691-6a28-460c-99e1-8eee48d43cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:29:34.459536 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 747/1494] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:29:34.467929 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876cd030-7961-4f48-810a-80daca2bb4e8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:29:34.468420 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876cd030-7961-4f48-810a-80daca2bb4e8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:29:34.468524 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876cd030-7961-4f48-810a-80daca2bb4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:29:34.486117 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-876cd030-7961-4f48-810a-80daca2bb4e8 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:29:34.487344 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-876cd030-7961-4f48-810a-80daca2bb4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:29:34.495137 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 748/1495] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:29:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:29:34.499613 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ff691-6a28-460c-99e1-8eee48d43cdb None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:29:34.509568 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ff691-6a28-460c-99e1-8eee48d43cdb None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:29:34.514219 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbff68f-b5f1-47b5-972d-6271730b751e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:29:34.514903 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbff68f-b5f1-47b5-972d-6271730b751e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:29:34.515192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbff68f-b5f1-47b5-972d-6271730b751e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:29:34.515474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbff68f-b5f1-47b5-972d-6271730b751e admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:29:34.516203 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbff68f-b5f1-47b5-972d-6271730b751e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:29:34.516512 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbff68f-b5f1-47b5-972d-6271730b751e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:29:34.517525 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-1fbff68f-b5f1-47b5-972d-6271730b751e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 29 12:29:34.517525 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jun 29 12:29:34.517525 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: Jun 29 12:29:34.518407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbff68f-b5f1-47b5-972d-6271730b751e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:29:34.530806 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 748/1496] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:29:34.538608 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876cd030-7961-4f48-810a-80daca2bb4e8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:29:34.547877 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876cd030-7961-4f48-810a-80daca2bb4e8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:29:34.552382 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62114b75-c40e-41ee-891c-d0b9f893e416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:29:34.553029 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62114b75-c40e-41ee-891c-d0b9f893e416 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:29:34.553320 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62114b75-c40e-41ee-891c-d0b9f893e416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:29:34.553604 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62114b75-c40e-41ee-891c-d0b9f893e416 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:29:34.554307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62114b75-c40e-41ee-891c-d0b9f893e416 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:29:34.556335 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-62114b75-c40e-41ee-891c-d0b9f893e416 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 12:29:34.556335 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jun 29 12:29:34.556335 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: Jun 29 12:29:34.557561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62114b75-c40e-41ee-891c-d0b9f893e416 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:29:34.598050 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 749/1497] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:29:34 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:29:34.602340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbff68f-b5f1-47b5-972d-6271730b751e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:29:34.612773 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbff68f-b5f1-47b5-972d-6271730b751e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:29:34.670021 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb3ae4-96a8-492d-b395-b64c95c6bf79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:29:34.670737 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb3ae4-96a8-492d-b395-b64c95c6bf79 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:29:34.671026 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb3ae4-96a8-492d-b395-b64c95c6bf79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:29:34.671307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb3ae4-96a8-492d-b395-b64c95c6bf79 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:29:34.672014 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb3ae4-96a8-492d-b395-b64c95c6bf79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:29:34.673023 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-a6cb3ae4-96a8-492d-b395-b64c95c6bf79 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 12:29:34.673023 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jun 29 12:29:34.673023 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: Jun 29 12:29:34.674218 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb3ae4-96a8-492d-b395-b64c95c6bf79 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:29:34.693068 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 749/1498] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:29:34 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:29:34.697708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62114b75-c40e-41ee-891c-d0b9f893e416 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:29:34.707641 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62114b75-c40e-41ee-891c-d0b9f893e416 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:29:34.712661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8372b80-7631-4a42-bdc8-f3bc33bfca86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:29:34.713324 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8372b80-7631-4a42-bdc8-f3bc33bfca86 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:29:34.713606 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8372b80-7631-4a42-bdc8-f3bc33bfca86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:29:34.713882 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8372b80-7631-4a42-bdc8-f3bc33bfca86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:29:34.714637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8372b80-7631-4a42-bdc8-f3bc33bfca86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:29:34.715644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-a8372b80-7631-4a42-bdc8-f3bc33bfca86 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 29 12:29:34.715644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jun 29 12:29:34.715644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: Jun 29 12:29:34.716516 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8372b80-7631-4a42-bdc8-f3bc33bfca86 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:29:34.728854 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 750/1499] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:29:34.732679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb3ae4-96a8-492d-b395-b64c95c6bf79 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:29:34.742656 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb3ae4-96a8-492d-b395-b64c95c6bf79 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:29:34.747347 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a17bdd-f3c9-4bff-a7c2-dd0fbf8fb539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:29:34.748092 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a17bdd-f3c9-4bff-a7c2-dd0fbf8fb539 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:29:34.748381 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a17bdd-f3c9-4bff-a7c2-dd0fbf8fb539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:29:34.748664 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a17bdd-f3c9-4bff-a7c2-dd0fbf8fb539 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:29:34.749518 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a17bdd-f3c9-4bff-a7c2-dd0fbf8fb539 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:29:34.750565 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-a3a17bdd-f3c9-4bff-a7c2-dd0fbf8fb539 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 29 12:29:34.750565 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jun 29 12:29:34.750565 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: Jun 29 12:29:34.751469 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a17bdd-f3c9-4bff-a7c2-dd0fbf8fb539 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:29:34.765105 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 750/1500] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:29:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:29:34.769128 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8372b80-7631-4a42-bdc8-f3bc33bfca86 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:29:34.780062 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8372b80-7631-4a42-bdc8-f3bc33bfca86 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:29:34.784543 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedafb6-829e-4726-96b0-2177d67f84db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:29:34.785420 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedafb6-829e-4726-96b0-2177d67f84db admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:29:34.785701 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedafb6-829e-4726-96b0-2177d67f84db admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:29:34.785975 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedafb6-829e-4726-96b0-2177d67f84db admin admin] PATH_INFO: `/v3/projects/d800c6c907b5435996a833765cf28f69/users/e3a2400ec58c4bd880b98a480f5f5804/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:29:34.802709 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dedafb6-829e-4726-96b0-2177d67f84db admin admin] RBAC: Authorizing `identity:create_grant(project_id=d800c6c907b5435996a833765cf28f69, user_id=e3a2400ec58c4bd880b98a480f5f5804, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:29:34.803814 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-0dedafb6-829e-4726-96b0-2177d67f84db admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 12:29:34.803814 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jun 29 12:29:34.803814 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: Jun 29 12:29:34.806285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dedafb6-829e-4726-96b0-2177d67f84db admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:29:34.819126 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 751/1501] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:29:34 2022] PUT /identity/v3/projects/d800c6c907b5435996a833765cf28f69/users/e3a2400ec58c4bd880b98a480f5f5804/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:29:34.831892 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11806bc-b64d-405f-8698-afd2cfb2e0ee None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:29:34.832190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11806bc-b64d-405f-8698-afd2cfb2e0ee None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:29:34.832479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11806bc-b64d-405f-8698-afd2cfb2e0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:29:34.858822 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c11806bc-b64d-405f-8698-afd2cfb2e0ee None None] MFA Rules not processed for user `e3a2400ec58c4bd880b98a480f5f5804`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:29:34.860041 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c11806bc-b64d-405f-8698-afd2cfb2e0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:29:34.929157 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 751/1502] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:29:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:29:34.942913 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d20fb7-43f8-4d7d-8828-79bf02ea3bb2 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:29:34.943214 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d20fb7-43f8-4d7d-8828-79bf02ea3bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:29:34.943503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d20fb7-43f8-4d7d-8828-79bf02ea3bb2 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:29:34.944865 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 752/1503] 10.222.0.52 () {54 vars in 963 bytes} [Wed Jun 29 12:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:29:34.950255 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391a9ba-4ca6-4dc1-b9f3-71911a3470a3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:29:34.950593 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391a9ba-4ca6-4dc1-b9f3-71911a3470a3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:29:34.950884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391a9ba-4ca6-4dc1-b9f3-71911a3470a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:29:34.968366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8391a9ba-4ca6-4dc1-b9f3-71911a3470a3 None None] MFA Rules not processed for user `356d396ae5cf47c3bc61f24f9e6dfa02`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:29:34.969558 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8391a9ba-4ca6-4dc1-b9f3-71911a3470a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:29:35.012681 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:29:35.021639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 752/1504] 10.222.0.52 () {58 vars in 1060 bytes} [Wed Jun 29 12:29:34 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:29:35.028936 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d20fb7-43f8-4d7d-8828-79bf02ea3bb2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:29:35.039884 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d20fb7-43f8-4d7d-8828-79bf02ea3bb2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:29:35.043672 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c55c1-382e-4f65-a652-fbef51f99c42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:29:35.044316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c55c1-382e-4f65-a652-fbef51f99c42 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:29:35.044605 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c55c1-382e-4f65-a652-fbef51f99c42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:29:35.044885 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c55c1-382e-4f65-a652-fbef51f99c42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:29:35.053137 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c55c1-382e-4f65-a652-fbef51f99c42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:29:35.054599 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c55c1-382e-4f65-a652-fbef51f99c42 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:29:35.063923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 753/1505] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:29:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:29:35.105855 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115358de-eb68-4612-8ffd-381154ccb3b9 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:29:35.106144 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115358de-eb68-4612-8ffd-381154ccb3b9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:29:35.106459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115358de-eb68-4612-8ffd-381154ccb3b9 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:29:35.107754 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 753/1506] 10.222.0.52 () {52 vars in 829 bytes} [Wed Jun 29 12:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:29:35.111304 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c55c1-382e-4f65-a652-fbef51f99c42 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:29:35.173377 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c55c1-382e-4f65-a652-fbef51f99c42 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:29:35.177874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6ecfc-568f-4cb7-abc8-c19bb87fd9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:29:35.178776 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6ecfc-568f-4cb7-abc8-c19bb87fd9fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:29:35.179071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6ecfc-568f-4cb7-abc8-c19bb87fd9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:29:35.179403 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6ecfc-568f-4cb7-abc8-c19bb87fd9fe admin admin] PATH_INFO: `/v3/users/e3a2400ec58c4bd880b98a480f5f5804` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:29:35.180535 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a6ecfc-568f-4cb7-abc8-c19bb87fd9fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3a2400ec58c4bd880b98a480f5f5804)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:29:35.182119 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-05a6ecfc-568f-4cb7-abc8-c19bb87fd9fe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 12:29:35.182119 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jun 29 12:29:35.182119 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: Jun 29 12:29:35.183361 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a6ecfc-568f-4cb7-abc8-c19bb87fd9fe admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:29:35.248005 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-05a6ecfc-568f-4cb7-abc8-c19bb87fd9fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3a2400ec58c4bd880b98a480f5f5804'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:29:35.248650 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-05a6ecfc-568f-4cb7-abc8-c19bb87fd9fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3a2400ec58c4bd880b98a480f5f5804'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:29:35.266371 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-05a6ecfc-568f-4cb7-abc8-c19bb87fd9fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3a2400ec58c4bd880b98a480f5f5804'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:29:35.267155 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-05a6ecfc-568f-4cb7-abc8-c19bb87fd9fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3a2400ec58c4bd880b98a480f5f5804'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:29:35.279428 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-05a6ecfc-568f-4cb7-abc8-c19bb87fd9fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3a2400ec58c4bd880b98a480f5f5804'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:29:35.283347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:29:35.295799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 754/1507] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:29:35 2022] DELETE /identity/v3/users/e3a2400ec58c4bd880b98a480f5f5804 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:29:35.300239 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0fa84-9c12-4fe2-8c5d-97b1a81e029c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:29:35.300526 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0fa84-9c12-4fe2-8c5d-97b1a81e029c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:29:35.300805 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b0fa84-9c12-4fe2-8c5d-97b1a81e029c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:29:35.319612 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-34b0fa84-9c12-4fe2-8c5d-97b1a81e029c None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:29:35.320788 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-34b0fa84-9c12-4fe2-8c5d-97b1a81e029c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:29:35.382301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 754/1508] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:29:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:29:35.391747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6ecfc-568f-4cb7-abc8-c19bb87fd9fe admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:29:35.393138 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-05a6ecfc-568f-4cb7-abc8-c19bb87fd9fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:29:35.450289 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6ecfc-568f-4cb7-abc8-c19bb87fd9fe admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:29:35.453754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc58d48-6f8d-4a40-82a1-cf409e918367 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:29:35.454399 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc58d48-6f8d-4a40-82a1-cf409e918367 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:29:35.454712 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc58d48-6f8d-4a40-82a1-cf409e918367 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:29:35.454992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc58d48-6f8d-4a40-82a1-cf409e918367 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:29:35.463777 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc58d48-6f8d-4a40-82a1-cf409e918367 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:29:35.465222 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc58d48-6f8d-4a40-82a1-cf409e918367 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:29:35.474401 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 755/1509] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:29:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:29:35.895305 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0fa84-9c12-4fe2-8c5d-97b1a81e029c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:29:35.897161 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-34b0fa84-9c12-4fe2-8c5d-97b1a81e029c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:29:35.909216 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b0fa84-9c12-4fe2-8c5d-97b1a81e029c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:29:35.913795 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13635b10-d535-4a0c-8c56-71998e9d2130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:29:35.914683 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13635b10-d535-4a0c-8c56-71998e9d2130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:29:35.914966 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13635b10-d535-4a0c-8c56-71998e9d2130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:29:35.915892 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13635b10-d535-4a0c-8c56-71998e9d2130 admin admin] PATH_INFO: `/v3/projects/d800c6c907b5435996a833765cf28f69` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:29:35.916978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13635b10-d535-4a0c-8c56-71998e9d2130 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d800c6c907b5435996a833765cf28f69)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:29:35.918141 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-13635b10-d535-4a0c-8c56-71998e9d2130 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 12:29:35.918141 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jun 29 12:29:35.918141 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: Jun 29 12:29:35.919192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13635b10-d535-4a0c-8c56-71998e9d2130 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:29:35.977111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-13635b10-d535-4a0c-8c56-71998e9d2130 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd800c6c907b5435996a833765cf28f69', 'request_id': 'req-13635b10-d535-4a0c-8c56-71998e9d2130'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:29:35.990731 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-13635b10-d535-4a0c-8c56-71998e9d2130 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd800c6c907b5435996a833765cf28f69', 'request_id': 'req-13635b10-d535-4a0c-8c56-71998e9d2130'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:29:35.997487 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-13635b10-d535-4a0c-8c56-71998e9d2130 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd800c6c907b5435996a833765cf28f69', 'request_id': 'req-13635b10-d535-4a0c-8c56-71998e9d2130'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:29:36.012754 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-13635b10-d535-4a0c-8c56-71998e9d2130 admin admin] The token cache is being invalidate because project d800c6c907b5435996a833765cf28f69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:29:36.013045 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-13635b10-d535-4a0c-8c56-71998e9d2130 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:29:36.013752 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-13635b10-d535-4a0c-8c56-71998e9d2130 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:29:36.015330 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 755/1510] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:29:35 2022] DELETE /identity/v3/projects/d800c6c907b5435996a833765cf28f69 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:29:45.470369 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:29:46.009622 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:31:14.211508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6d993-a8c9-4b71-a950-c747f9d6b72c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:14.212147 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6d993-a8c9-4b71-a950-c747f9d6b72c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:14.212258 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6d993-a8c9-4b71-a950-c747f9d6b72c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:14.213624 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 12:31:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:31:14.218689 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b6828-54d3-4483-b047-c45305de1db7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:14.219128 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b6828-54d3-4483-b047-c45305de1db7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:14.219262 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740b6828-54d3-4483-b047-c45305de1db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:14.237162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-740b6828-54d3-4483-b047-c45305de1db7 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:31:14.238369 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-740b6828-54d3-4483-b047-c45305de1db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:31:14.298428 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 12:31:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:31:14.456686 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f6d993-a8c9-4b71-a950-c747f9d6b72c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:31:14.458349 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-19f6d993-a8c9-4b71-a950-c747f9d6b72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:31:14.532654 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f6d993-a8c9-4b71-a950-c747f9d6b72c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:31:14.538039 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e8a4a-7121-48a0-b727-94b1dee5aac7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:31:14.539342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e8a4a-7121-48a0-b727-94b1dee5aac7 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:14.539626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e8a4a-7121-48a0-b727-94b1dee5aac7 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:14.539899 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e8a4a-7121-48a0-b727-94b1dee5aac7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:14.548164 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e8a4a-7121-48a0-b727-94b1dee5aac7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:31:14.550042 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e8a4a-7121-48a0-b727-94b1dee5aac7 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:31:14.559477 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 757/1513] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:31:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:31:15.394305 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169b1cd-5694-49f8-aa63-201d3b40fb61 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:15.394821 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169b1cd-5694-49f8-aa63-201d3b40fb61 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:15.394927 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169b1cd-5694-49f8-aa63-201d3b40fb61 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:15.396524 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 12:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:31:15.402032 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400c967-0e68-423c-8b04-797fb4397062 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:15.402479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400c967-0e68-423c-8b04-797fb4397062 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:15.402647 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6400c967-0e68-423c-8b04-797fb4397062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:15.423146 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6400c967-0e68-423c-8b04-797fb4397062 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:31:15.424358 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6400c967-0e68-423c-8b04-797fb4397062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:31:15.432254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 12:31:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:31:15.583076 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c169b1cd-5694-49f8-aa63-201d3b40fb61 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:31:15.598160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c169b1cd-5694-49f8-aa63-201d3b40fb61 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:31:15.604421 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392174f5-10fd-4322-8d6c-c6b802ca100d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:31:15.605068 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392174f5-10fd-4322-8d6c-c6b802ca100d service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:15.605354 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392174f5-10fd-4322-8d6c-c6b802ca100d service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:15.605634 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392174f5-10fd-4322-8d6c-c6b802ca100d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:15.614494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392174f5-10fd-4322-8d6c-c6b802ca100d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:31:15.616494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392174f5-10fd-4322-8d6c-c6b802ca100d service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:31:15.627554 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 758/1516] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:31:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:31:16.235163 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f78fb-9a72-4499-b8cb-a798a00eb364 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:16.235513 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f78fb-9a72-4499-b8cb-a798a00eb364 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:16.235715 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f78fb-9a72-4499-b8cb-a798a00eb364 None None] PATH_INFO: `/v3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:16.237011 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 759/1517] 10.222.0.23 () {54 vars in 925 bytes} [Wed Jun 29 12:31:16 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:31:16.243603 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12c3dd-e55a-4df9-bfc1-4b1aab946b68 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:16.243902 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12c3dd-e55a-4df9-bfc1-4b1aab946b68 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:16.244217 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12c3dd-e55a-4df9-bfc1-4b1aab946b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:16.262601 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3a12c3dd-e55a-4df9-bfc1-4b1aab946b68 None None] MFA Rules not processed for user `c98135cacb9d4f22846ad33500b9bd1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:31:16.263844 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3a12c3dd-e55a-4df9-bfc1-4b1aab946b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:31:16.327553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 759/1518] 10.222.0.23 () {58 vars in 1015 bytes} [Wed Jun 29 12:31:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:31:20.073519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb787eb-d09f-4d69-8732-80d4739e8d8f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:20.074064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb787eb-d09f-4d69-8732-80d4739e8d8f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:20.074064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb787eb-d09f-4d69-8732-80d4739e8d8f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:20.075616 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 760/1519] 10.222.0.52 () {54 vars in 960 bytes} [Wed Jun 29 12:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:31:20.080533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8632667b-0efa-42ce-81d8-effdc1494a25 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:20.080982 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8632667b-0efa-42ce-81d8-effdc1494a25 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:20.081135 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8632667b-0efa-42ce-81d8-effdc1494a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:20.099144 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8632667b-0efa-42ce-81d8-effdc1494a25 None None] MFA Rules not processed for user `c98135cacb9d4f22846ad33500b9bd1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:31:20.100341 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8632667b-0efa-42ce-81d8-effdc1494a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:31:20.109434 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 760/1520] 10.222.0.52 () {58 vars in 1057 bytes} [Wed Jun 29 12:31:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:31:20.117528 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb787eb-d09f-4d69-8732-80d4739e8d8f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:31:20.130696 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb787eb-d09f-4d69-8732-80d4739e8d8f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:31:20.135925 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb606bc-3e18-4e1e-85e1-298b5a10c9a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98135cacb9d4f22846ad33500b9bd1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:31:20.136564 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb606bc-3e18-4e1e-85e1-298b5a10c9a6 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:20.136852 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb606bc-3e18-4e1e-85e1-298b5a10c9a6 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:20.137132 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb606bc-3e18-4e1e-85e1-298b5a10c9a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:20.145351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb606bc-3e18-4e1e-85e1-298b5a10c9a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:31:20.147241 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb606bc-3e18-4e1e-85e1-298b5a10c9a6 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:31:20.157042 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 761/1521] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:31:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:31:26.936721 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7d1cc-a274-4517-8b31-2d6d5ca2342e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:26.937278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7d1cc-a274-4517-8b31-2d6d5ca2342e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:26.937278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7d1cc-a274-4517-8b31-2d6d5ca2342e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:26.938697 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 12:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:31:26.943648 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0931c8-a605-4f2b-8895-37a9a0a6175f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:26.944036 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0931c8-a605-4f2b-8895-37a9a0a6175f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:26.944199 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0931c8-a605-4f2b-8895-37a9a0a6175f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:26.962863 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ac0931c8-a605-4f2b-8895-37a9a0a6175f None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:31:26.964046 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ac0931c8-a605-4f2b-8895-37a9a0a6175f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:31:26.972291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 12:31:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:31:27.127034 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7d1cc-a274-4517-8b31-2d6d5ca2342e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:31:27.141615 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7d1cc-a274-4517-8b31-2d6d5ca2342e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:31:27.147162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eee459-3960-4170-b474-7a3d661f4ee5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:31:27.148429 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eee459-3960-4170-b474-7a3d661f4ee5 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:27.148714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eee459-3960-4170-b474-7a3d661f4ee5 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:27.148992 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eee459-3960-4170-b474-7a3d661f4ee5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:27.156932 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eee459-3960-4170-b474-7a3d661f4ee5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:31:27.158554 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eee459-3960-4170-b474-7a3d661f4ee5 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:31:27.168104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 762/1524] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:31:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:31:27.977670 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a720a3-0d35-4b80-ab8b-473aeaa5d47a None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:27.978255 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a720a3-0d35-4b80-ab8b-473aeaa5d47a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:27.978255 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a720a3-0d35-4b80-ab8b-473aeaa5d47a None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:27.979644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jun 29 12:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:31:27.984329 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5488f-2412-467d-9185-692a07744fe2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:27.984800 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5488f-2412-467d-9185-692a07744fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:27.984916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5488f-2412-467d-9185-692a07744fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:28.003857 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-41c5488f-2412-467d-9185-692a07744fe2 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:31:28.005057 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-41c5488f-2412-467d-9185-692a07744fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:31:28.013833 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jun 29 12:31:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:31:28.162973 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a720a3-0d35-4b80-ab8b-473aeaa5d47a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:31:28.176997 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a720a3-0d35-4b80-ab8b-473aeaa5d47a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:31:28.182909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87700f-585d-41c2-837b-c7b3992cff79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:31:28.183539 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87700f-585d-41c2-837b-c7b3992cff79 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:28.183829 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87700f-585d-41c2-837b-c7b3992cff79 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:28.184107 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87700f-585d-41c2-837b-c7b3992cff79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:28.192082 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de87700f-585d-41c2-837b-c7b3992cff79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:31:28.193648 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de87700f-585d-41c2-837b-c7b3992cff79 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:31:28.203498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 764/1527] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:31:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:31:32.208508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7698a16-82d0-4892-acd7-9e3f3faf660a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:32.209099 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7698a16-82d0-4892-acd7-9e3f3faf660a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:32.209099 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7698a16-82d0-4892-acd7-9e3f3faf660a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:32.210496 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 12:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:31:32.216113 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89324a45-3951-434a-8ea0-d22057d773eb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:32.216526 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89324a45-3951-434a-8ea0-d22057d773eb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:32.216678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89324a45-3951-434a-8ea0-d22057d773eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:32.234763 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-89324a45-3951-434a-8ea0-d22057d773eb None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:31:32.235942 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-89324a45-3951-434a-8ea0-d22057d773eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:31:32.244226 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 12:31:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:31:32.280382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7698a16-82d0-4892-acd7-9e3f3faf660a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:31:32.282047 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a7698a16-82d0-4892-acd7-9e3f3faf660a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:31:32.339462 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7698a16-82d0-4892-acd7-9e3f3faf660a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:31:32.343021 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9e2c6-f447-4e0d-8436-003810fa40c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:31:32.343665 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9e2c6-f447-4e0d-8436-003810fa40c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:32.343956 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9e2c6-f447-4e0d-8436-003810fa40c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:32.344236 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9e2c6-f447-4e0d-8436-003810fa40c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:32.354243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e9e2c6-f447-4e0d-8436-003810fa40c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:31:32.355723 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e9e2c6-f447-4e0d-8436-003810fa40c9 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:31:32.360816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:31:32.366440 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 765/1530] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:31:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:31:34.411429 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b8675-3ea9-40de-9867-54889f3efdc8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:34.411946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b8675-3ea9-40de-9867-54889f3efdc8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:34.412067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b8675-3ea9-40de-9867-54889f3efdc8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:34.413362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 12:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:31:34.418678 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec338990-3259-4298-a44a-f64758ed1196 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:34.419109 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec338990-3259-4298-a44a-f64758ed1196 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:34.419250 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec338990-3259-4298-a44a-f64758ed1196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:34.438582 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ec338990-3259-4298-a44a-f64758ed1196 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:31:34.439782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ec338990-3259-4298-a44a-f64758ed1196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:31:34.448069 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 12:31:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:31:34.484260 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b8675-3ea9-40de-9867-54889f3efdc8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:31:34.494996 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b8675-3ea9-40de-9867-54889f3efdc8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:31:34.498472 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d180ec68-96a7-4a41-9e8e-f65e79695ee7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:31:34.499116 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180ec68-96a7-4a41-9e8e-f65e79695ee7 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:34.499404 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180ec68-96a7-4a41-9e8e-f65e79695ee7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:34.499683 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d180ec68-96a7-4a41-9e8e-f65e79695ee7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:34.509065 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d180ec68-96a7-4a41-9e8e-f65e79695ee7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:31:34.510511 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d180ec68-96a7-4a41-9e8e-f65e79695ee7 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:31:34.519781 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 767/1533] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:31:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:31:38.349544 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154b31d-3639-447e-aac9-72fd60071d79 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:38.350121 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154b31d-3639-447e-aac9-72fd60071d79 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:38.350121 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154b31d-3639-447e-aac9-72fd60071d79 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:38.351503 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 12:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:31:38.357401 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a2848-d302-4d9b-a239-c19d3d811b2c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:38.357861 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a2848-d302-4d9b-a239-c19d3d811b2c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:38.357961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a2848-d302-4d9b-a239-c19d3d811b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:38.364428 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=62 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:31:38.377789 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a85a2848-d302-4d9b-a239-c19d3d811b2c None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:31:38.379017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a85a2848-d302-4d9b-a239-c19d3d811b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:31:38.386801 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 12:31:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:31:38.693782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154b31d-3639-447e-aac9-72fd60071d79 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:31:38.695492 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5154b31d-3639-447e-aac9-72fd60071d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:31:38.752996 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154b31d-3639-447e-aac9-72fd60071d79 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:31:38.756499 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e12cf-2a76-4738-a4ee-467a7397b0d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:31:38.757135 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e12cf-2a76-4738-a4ee-467a7397b0d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:38.757423 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e12cf-2a76-4738-a4ee-467a7397b0d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:38.757710 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e12cf-2a76-4738-a4ee-467a7397b0d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:38.767223 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6e12cf-2a76-4738-a4ee-467a7397b0d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:31:38.768654 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6e12cf-2a76-4738-a4ee-467a7397b0d4 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:31:38.778266 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 768/1536] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:31:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:31:40.863409 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455b3dc-9c93-4025-b6e9-c6d1a7947421 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:40.863806 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455b3dc-9c93-4025-b6e9-c6d1a7947421 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:40.864005 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8455b3dc-9c93-4025-b6e9-c6d1a7947421 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:40.865269 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jun 29 12:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:31:40.871019 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eea2df-f235-4c3c-8031-ac24fe690867 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:40.871430 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eea2df-f235-4c3c-8031-ac24fe690867 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:40.871583 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eea2df-f235-4c3c-8031-ac24fe690867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:40.889741 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d6eea2df-f235-4c3c-8031-ac24fe690867 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:31:40.890974 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d6eea2df-f235-4c3c-8031-ac24fe690867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:31:40.899051 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jun 29 12:31:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:31:41.208648 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455b3dc-9c93-4025-b6e9-c6d1a7947421 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:31:41.210448 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8455b3dc-9c93-4025-b6e9-c6d1a7947421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:31:41.223679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8455b3dc-9c93-4025-b6e9-c6d1a7947421 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:31:41.227910 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09957381-0b79-4493-beef-d12e835936fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:31:41.228550 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09957381-0b79-4493-beef-d12e835936fb service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:31:41.228837 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09957381-0b79-4493-beef-d12e835936fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:31:41.229120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09957381-0b79-4493-beef-d12e835936fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:31:41.237929 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09957381-0b79-4493-beef-d12e835936fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:31:41.239383 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09957381-0b79-4493-beef-d12e835936fb service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:31:41.249412 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 770/1539] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:31:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:31:50.884593 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=32 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:31:51.245409 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=8 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:09.845769 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e263f-b227-4e3e-9e1c-736c424e2f29 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:09.846476 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e263f-b227-4e3e-9e1c-736c424e2f29 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:09.846476 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631e263f-b227-4e3e-9e1c-736c424e2f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:09.856977 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d39a79-d73c-495a-a020-2aa2221f1357 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:09.857638 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d39a79-d73c-495a-a020-2aa2221f1357 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:09.857638 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d39a79-d73c-495a-a020-2aa2221f1357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:09.885383 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-631e263f-b227-4e3e-9e1c-736c424e2f29 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:09.886607 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-631e263f-b227-4e3e-9e1c-736c424e2f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:09.889546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c1d39a79-d73c-495a-a020-2aa2221f1357 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:09.890806 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c1d39a79-d73c-495a-a020-2aa2221f1357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:09.908321 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 771/1541] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:33:09.908554 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 770/1541] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:09.917753 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef704d83-4855-4629-83d8-62c817d37289 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:09.917968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c554d-23f1-49da-adab-afe3057166c0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:09.918096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef704d83-4855-4629-83d8-62c817d37289 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:09.918184 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c554d-23f1-49da-adab-afe3057166c0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:09.918453 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef704d83-4855-4629-83d8-62c817d37289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:09.918574 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c554d-23f1-49da-adab-afe3057166c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:09.951898 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ef704d83-4855-4629-83d8-62c817d37289 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:09.952139 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-083c554d-23f1-49da-adab-afe3057166c0 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:09.953127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ef704d83-4855-4629-83d8-62c817d37289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:09.953264 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-083c554d-23f1-49da-adab-afe3057166c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:09.968351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 772/1542] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:33:09.973034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c554d-23f1-49da-adab-afe3057166c0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:09.980293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 771/1543] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:09.984496 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef704d83-4855-4629-83d8-62c817d37289 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:09.990713 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c554d-23f1-49da-adab-afe3057166c0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:09.999481 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228fe7aa-2d17-41f7-8c04-c609f2dc7722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:10.000221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fe7aa-2d17-41f7-8c04-c609f2dc7722 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.000552 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fe7aa-2d17-41f7-8c04-c609f2dc7722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.000931 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228fe7aa-2d17-41f7-8c04-c609f2dc7722 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.002021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228fe7aa-2d17-41f7-8c04-c609f2dc7722 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:33:10.002366 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228fe7aa-2d17-41f7-8c04-c609f2dc7722 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:10.003221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef704d83-4855-4629-83d8-62c817d37289 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:10.003511 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-228fe7aa-2d17-41f7-8c04-c609f2dc7722 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 29 12:33:10.003511 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jun 29 12:33:10.003511 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: Jun 29 12:33:10.004439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228fe7aa-2d17-41f7-8c04-c609f2dc7722 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:10.027736 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3dafd7-74a2-49d4-90fb-ee18de03acd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:10.028443 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3dafd7-74a2-49d4-90fb-ee18de03acd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.028733 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3dafd7-74a2-49d4-90fb-ee18de03acd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.029367 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3dafd7-74a2-49d4-90fb-ee18de03acd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.030115 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3dafd7-74a2-49d4-90fb-ee18de03acd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:33:10.030455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3dafd7-74a2-49d4-90fb-ee18de03acd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:10.031697 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 773/1544] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:33:10.037168 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3dafd7-74a2-49d4-90fb-ee18de03acd7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:10.040853 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228fe7aa-2d17-41f7-8c04-c609f2dc7722 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:10.055723 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228fe7aa-2d17-41f7-8c04-c609f2dc7722 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:10.063304 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 772/1545] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:33:10.070493 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3dafd7-74a2-49d4-90fb-ee18de03acd7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:10.070765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee7dbc-f9c1-47fb-b75d-edcee9c516df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:10.071392 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee7dbc-f9c1-47fb-b75d-edcee9c516df admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.071685 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee7dbc-f9c1-47fb-b75d-edcee9c516df admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.071970 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee7dbc-f9c1-47fb-b75d-edcee9c516df admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.072680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ee7dbc-f9c1-47fb-b75d-edcee9c516df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:10.074621 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ee7dbc-f9c1-47fb-b75d-edcee9c516df admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:10.086905 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3dafd7-74a2-49d4-90fb-ee18de03acd7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:10.098123 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf2d453-22c0-461b-85bd-e73977ecfb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:10.099294 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2d453-22c0-461b-85bd-e73977ecfb40 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.099719 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2d453-22c0-461b-85bd-e73977ecfb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.100290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2d453-22c0-461b-85bd-e73977ecfb40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.100628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf2d453-22c0-461b-85bd-e73977ecfb40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:10.101677 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-cdf2d453-22c0-461b-85bd-e73977ecfb40 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 12:33:10.101677 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jun 29 12:33:10.101677 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: Jun 29 12:33:10.103070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf2d453-22c0-461b-85bd-e73977ecfb40 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:10.134042 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 774/1546] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:10 2022] POST /identity/v3/projects => generated 387 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:33:10.135637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7711fa8-ce4c-400e-adf5-bd9bd95bde88 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.136120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7711fa8-ce4c-400e-adf5-bd9bd95bde88 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.136492 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7711fa8-ce4c-400e-adf5-bd9bd95bde88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.151019 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 773/1547] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:10 2022] POST /identity/v3/projects => generated 365 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:33:10.152955 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318baf9-aa4f-4cda-845c-6f30121058c6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.154674 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318baf9-aa4f-4cda-845c-6f30121058c6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.155004 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b318baf9-aa4f-4cda-845c-6f30121058c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.173670 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f7711fa8-ce4c-400e-adf5-bd9bd95bde88 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:10.175034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f7711fa8-ce4c-400e-adf5-bd9bd95bde88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:10.187493 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b318baf9-aa4f-4cda-845c-6f30121058c6 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:10.189086 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b318baf9-aa4f-4cda-845c-6f30121058c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:10.246936 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 775/1548] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:10.247779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7711fa8-ce4c-400e-adf5-bd9bd95bde88 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:10.257796 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7711fa8-ce4c-400e-adf5-bd9bd95bde88 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:10.259744 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 774/1549] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:10.260566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b318baf9-aa4f-4cda-845c-6f30121058c6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:10.262471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e678c2-302f-4730-92bd-ea062951224a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:10.263144 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e678c2-302f-4730-92bd-ea062951224a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.263434 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e678c2-302f-4730-92bd-ea062951224a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.263716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e678c2-302f-4730-92bd-ea062951224a admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.264412 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e678c2-302f-4730-92bd-ea062951224a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:10.265441 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-64e678c2-302f-4730-92bd-ea062951224a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 12:33:10.265441 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jun 29 12:33:10.265441 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: Jun 29 12:33:10.266958 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e678c2-302f-4730-92bd-ea062951224a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:10.270168 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b318baf9-aa4f-4cda-845c-6f30121058c6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:10.275595 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037d0db-edee-4b9c-af7d-3cb82f74ae92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:10.276261 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037d0db-edee-4b9c-af7d-3cb82f74ae92 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.276547 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037d0db-edee-4b9c-af7d-3cb82f74ae92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.276852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037d0db-edee-4b9c-af7d-3cb82f74ae92 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.277605 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3037d0db-edee-4b9c-af7d-3cb82f74ae92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:10.280085 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3037d0db-edee-4b9c-af7d-3cb82f74ae92 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:10.292063 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 776/1550] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:10 2022] POST /identity/v3/users => generated 294 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:10.292063 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8dfe5-f3d5-4661-9ce4-2e48c4d23680 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.292063 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8dfe5-f3d5-4661-9ce4-2e48c4d23680 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.292063 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8dfe5-f3d5-4661-9ce4-2e48c4d23680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.302777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 775/1551] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:10 2022] POST /identity/v3/users => generated 283 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:10.304156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754daa3b-b19b-468e-b3bb-89b94b257489 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.304448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754daa3b-b19b-468e-b3bb-89b94b257489 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.304735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754daa3b-b19b-468e-b3bb-89b94b257489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.313145 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-20c8dfe5-f3d5-4661-9ce4-2e48c4d23680 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:10.314650 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-20c8dfe5-f3d5-4661-9ce4-2e48c4d23680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:10.323794 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-754daa3b-b19b-468e-b3bb-89b94b257489 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:10.324320 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 777/1552] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:10.325198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-754daa3b-b19b-468e-b3bb-89b94b257489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:10.325577 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90b528-cba1-4e93-875e-d42a9649133d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.325869 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90b528-cba1-4e93-875e-d42a9649133d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.326152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90b528-cba1-4e93-875e-d42a9649133d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.334394 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 776/1553] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:10.339847 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32fd7f-17be-43c3-87dc-f9e30ff00e15 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.340290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32fd7f-17be-43c3-87dc-f9e30ff00e15 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.340494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c32fd7f-17be-43c3-87dc-f9e30ff00e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.345853 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-df90b528-cba1-4e93-875e-d42a9649133d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:10.347100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-df90b528-cba1-4e93-875e-d42a9649133d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:10.356914 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 778/1554] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:10.358151 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6eadc-22d7-4d7a-a2ec-743b115f4a87 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.358466 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6eadc-22d7-4d7a-a2ec-743b115f4a87 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.358758 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6eadc-22d7-4d7a-a2ec-743b115f4a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.362951 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5c32fd7f-17be-43c3-87dc-f9e30ff00e15 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:10.364284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5c32fd7f-17be-43c3-87dc-f9e30ff00e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:10.372711 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 777/1555] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:10.373950 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa5545-4805-4832-82ba-78f6df894cfa None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.374237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa5545-4805-4832-82ba-78f6df894cfa None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.374547 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaa5545-4805-4832-82ba-78f6df894cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.377840 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-87d6eadc-22d7-4d7a-a2ec-743b115f4a87 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:10.379063 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-87d6eadc-22d7-4d7a-a2ec-743b115f4a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:10.387566 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 779/1556] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:10.388369 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6eadc-22d7-4d7a-a2ec-743b115f4a87 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:10.394158 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-edaa5545-4805-4832-82ba-78f6df894cfa None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:10.395647 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-edaa5545-4805-4832-82ba-78f6df894cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:10.398014 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6eadc-22d7-4d7a-a2ec-743b115f4a87 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:10.402527 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe03b8-7cd0-48f0-81ee-86c065ab24aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:10.403179 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe03b8-7cd0-48f0-81ee-86c065ab24aa admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.403464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe03b8-7cd0-48f0-81ee-86c065ab24aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.403744 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe03b8-7cd0-48f0-81ee-86c065ab24aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.404041 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 778/1557] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:10.404476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe03b8-7cd0-48f0-81ee-86c065ab24aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:10.404846 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa5545-4805-4832-82ba-78f6df894cfa None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:10.406073 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe03b8-7cd0-48f0-81ee-86c065ab24aa admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:10.420062 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 780/1558] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:10.420237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaa5545-4805-4832-82ba-78f6df894cfa None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:10.420442 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7fb5f-39ea-4fc0-ab76-3c10d1492be0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.420442 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7fb5f-39ea-4fc0-ab76-3c10d1492be0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.420442 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7fb5f-39ea-4fc0-ab76-3c10d1492be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.422315 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d8a27-7e02-4b02-9b2d-e01a930a5a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:10.423040 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d8a27-7e02-4b02-9b2d-e01a930a5a1d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.423326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d8a27-7e02-4b02-9b2d-e01a930a5a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.423612 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d8a27-7e02-4b02-9b2d-e01a930a5a1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.424338 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d8a27-7e02-4b02-9b2d-e01a930a5a1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:10.425974 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d8a27-7e02-4b02-9b2d-e01a930a5a1d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:10.439604 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 779/1559] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:10.439802 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c3c7fb5f-39ea-4fc0-ab76-3c10d1492be0 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:10.439913 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69757834-9ff2-4527-ba81-53b39e7f25ca None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.439913 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69757834-9ff2-4527-ba81-53b39e7f25ca None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.440070 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c3c7fb5f-39ea-4fc0-ab76-3c10d1492be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:10.440147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69757834-9ff2-4527-ba81-53b39e7f25ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.447520 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 781/1560] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:10.448839 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5addf6e-e336-41fd-a778-c7ffc6611d48 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.449192 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5addf6e-e336-41fd-a778-c7ffc6611d48 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.449481 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5addf6e-e336-41fd-a778-c7ffc6611d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.456931 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-69757834-9ff2-4527-ba81-53b39e7f25ca None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:10.458156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-69757834-9ff2-4527-ba81-53b39e7f25ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:10.466475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 780/1561] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:10.467780 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee390747-edbd-445a-8659-27ea285097ad None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.468076 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee390747-edbd-445a-8659-27ea285097ad None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.468360 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee390747-edbd-445a-8659-27ea285097ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.468474 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f5addf6e-e336-41fd-a778-c7ffc6611d48 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:10.469599 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f5addf6e-e336-41fd-a778-c7ffc6611d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:10.478576 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 782/1562] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:10.479367 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5addf6e-e336-41fd-a778-c7ffc6611d48 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:10.487071 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ee390747-edbd-445a-8659-27ea285097ad None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:10.488251 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ee390747-edbd-445a-8659-27ea285097ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:10.488466 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5addf6e-e336-41fd-a778-c7ffc6611d48 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:10.493823 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49240c-feb1-486a-85de-dc48418ecaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:10.494490 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49240c-feb1-486a-85de-dc48418ecaaf admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.494781 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49240c-feb1-486a-85de-dc48418ecaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.495058 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d49240c-feb1-486a-85de-dc48418ecaaf admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.495779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d49240c-feb1-486a-85de-dc48418ecaaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:33:10.496090 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d49240c-feb1-486a-85de-dc48418ecaaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:10.496990 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 781/1563] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:10.497697 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d49240c-feb1-486a-85de-dc48418ecaaf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:10.497808 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee390747-edbd-445a-8659-27ea285097ad None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:10.517232 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 783/1564] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:33:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:33:10.517232 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49240c-feb1-486a-85de-dc48418ecaaf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:10.517538 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee390747-edbd-445a-8659-27ea285097ad None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:10.520859 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d49240c-feb1-486a-85de-dc48418ecaaf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:10.521963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b048f-eb6d-4312-b206-d93be9dd3fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:10.522653 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b048f-eb6d-4312-b206-d93be9dd3fdd admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.522970 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b048f-eb6d-4312-b206-d93be9dd3fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.523251 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b048f-eb6d-4312-b206-d93be9dd3fdd admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.524042 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b048f-eb6d-4312-b206-d93be9dd3fdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:33:10.524351 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b048f-eb6d-4312-b206-d93be9dd3fdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:10.526007 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b048f-eb6d-4312-b206-d93be9dd3fdd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:10.526134 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f39c68-ddae-4851-8ba4-d6807ac5dc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:10.526769 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f39c68-ddae-4851-8ba4-d6807ac5dc3d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.527112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f39c68-ddae-4851-8ba4-d6807ac5dc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.527393 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f39c68-ddae-4851-8ba4-d6807ac5dc3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.528139 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f39c68-ddae-4851-8ba4-d6807ac5dc3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:10.529841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f39c68-ddae-4851-8ba4-d6807ac5dc3d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:10.538919 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 782/1565] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:33:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:33:10.539922 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b048f-eb6d-4312-b206-d93be9dd3fdd admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:10.542140 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 784/1566] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:10.542983 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f39c68-ddae-4851-8ba4-d6807ac5dc3d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:10.551043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b048f-eb6d-4312-b206-d93be9dd3fdd admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:10.553740 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f39c68-ddae-4851-8ba4-d6807ac5dc3d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:10.556331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7348f-3cc5-4b81-8a76-a1aadbbcdda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:10.556990 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7348f-3cc5-4b81-8a76-a1aadbbcdda9 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.557270 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7348f-3cc5-4b81-8a76-a1aadbbcdda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.557554 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce7348f-3cc5-4b81-8a76-a1aadbbcdda9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.558270 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce7348f-3cc5-4b81-8a76-a1aadbbcdda9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:10.558376 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c5abb-1b9c-434d-bea1-82cf436333d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:10.559075 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c5abb-1b9c-434d-bea1-82cf436333d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.559363 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c5abb-1b9c-434d-bea1-82cf436333d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.559641 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c5abb-1b9c-434d-bea1-82cf436333d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.559935 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce7348f-3cc5-4b81-8a76-a1aadbbcdda9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:10.560355 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c5abb-1b9c-434d-bea1-82cf436333d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:33:10.560663 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c5abb-1b9c-434d-bea1-82cf436333d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:10.562224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c5abb-1b9c-434d-bea1-82cf436333d5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:10.571552 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 783/1567] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:10.572201 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7348f-3cc5-4b81-8a76-a1aadbbcdda9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:10.574911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 785/1568] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:33:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:33:10.575689 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c5abb-1b9c-434d-bea1-82cf436333d5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:10.583264 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce7348f-3cc5-4b81-8a76-a1aadbbcdda9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:10.585956 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c5abb-1b9c-434d-bea1-82cf436333d5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:10.587817 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f41bb-1d15-42cf-b7e2-75d49e09d609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:10.588467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f41bb-1d15-42cf-b7e2-75d49e09d609 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.588755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f41bb-1d15-42cf-b7e2-75d49e09d609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.589037 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f41bb-1d15-42cf-b7e2-75d49e09d609 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.589755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f41bb-1d15-42cf-b7e2-75d49e09d609 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:33:10.590136 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f41bb-1d15-42cf-b7e2-75d49e09d609 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:10.590617 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc2d4f-0dcc-4c45-9d55-2a36c9244f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:10.591258 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc2d4f-0dcc-4c45-9d55-2a36c9244f8d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.591546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc2d4f-0dcc-4c45-9d55-2a36c9244f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.591739 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f41bb-1d15-42cf-b7e2-75d49e09d609 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:10.591862 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc2d4f-0dcc-4c45-9d55-2a36c9244f8d admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.592525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bc2d4f-0dcc-4c45-9d55-2a36c9244f8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:33:10.592833 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bc2d4f-0dcc-4c45-9d55-2a36c9244f8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:10.594392 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bc2d4f-0dcc-4c45-9d55-2a36c9244f8d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:10.603776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 784/1569] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:33:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:33:10.604037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:10.604579 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f41bb-1d15-42cf-b7e2-75d49e09d609 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:10.608632 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 786/1570] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:33:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:33:10.609698 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc2d4f-0dcc-4c45-9d55-2a36c9244f8d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:10.615370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f41bb-1d15-42cf-b7e2-75d49e09d609 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:10.621485 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6b0e5-871c-48c1-8976-deeb56dd28b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:10.622143 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6b0e5-871c-48c1-8976-deeb56dd28b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.622457 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6b0e5-871c-48c1-8976-deeb56dd28b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.622748 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6b0e5-871c-48c1-8976-deeb56dd28b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.623472 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6b0e5-871c-48c1-8976-deeb56dd28b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:33:10.623783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6b0e5-871c-48c1-8976-deeb56dd28b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:10.625383 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6b0e5-871c-48c1-8976-deeb56dd28b5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:10.628162 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc2d4f-0dcc-4c45-9d55-2a36c9244f8d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:10.632744 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b60260-2575-4527-9ed5-f3b879420148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:10.633388 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b60260-2575-4527-9ed5-f3b879420148 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.633669 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b60260-2575-4527-9ed5-f3b879420148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.633946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b60260-2575-4527-9ed5-f3b879420148 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.634498 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:10.634670 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b60260-2575-4527-9ed5-f3b879420148 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:10.636549 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b60260-2575-4527-9ed5-f3b879420148 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:10.638628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 785/1571] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:33:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:33:10.639402 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6b0e5-871c-48c1-8976-deeb56dd28b5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:10.653045 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6b0e5-871c-48c1-8976-deeb56dd28b5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:10.657828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b594cb9-a836-4dfd-8a96-625d56107257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:10.658632 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b594cb9-a836-4dfd-8a96-625d56107257 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.658984 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b594cb9-a836-4dfd-8a96-625d56107257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.659325 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b594cb9-a836-4dfd-8a96-625d56107257 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.660073 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b594cb9-a836-4dfd-8a96-625d56107257 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:10.662811 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b594cb9-a836-4dfd-8a96-625d56107257 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:10.671106 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 787/1572] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:10 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:10.671106 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b60260-2575-4527-9ed5-f3b879420148 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:10.689007 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 786/1573] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:10 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:10.690042 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b594cb9-a836-4dfd-8a96-625d56107257 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:10.744311 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b60260-2575-4527-9ed5-f3b879420148 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:10.749262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349821e-52e3-4b27-a836-6ea02d9f5058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:10.750161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349821e-52e3-4b27-a836-6ea02d9f5058 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.750464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349821e-52e3-4b27-a836-6ea02d9f5058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.750744 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349821e-52e3-4b27-a836-6ea02d9f5058 admin admin] PATH_INFO: `/v3/projects/0ed882ae485441db99ccf6109be51b22/users/6252d0519ed9421e89710ca5294524c5/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.753372 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b594cb9-a836-4dfd-8a96-625d56107257 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:10.760013 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae8903b-3b57-4d43-8350-b898539da3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:10.760901 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8903b-3b57-4d43-8350-b898539da3ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.761186 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8903b-3b57-4d43-8350-b898539da3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.761463 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8903b-3b57-4d43-8350-b898539da3ed admin admin] PATH_INFO: `/v3/projects/1b65b9e1603349d2880eb2b4d793211e/users/aa9c3da38c7e44c2892b117c204ca7bc/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.766432 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349821e-52e3-4b27-a836-6ea02d9f5058 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ed882ae485441db99ccf6109be51b22, user_id=6252d0519ed9421e89710ca5294524c5, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:10.769121 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349821e-52e3-4b27-a836-6ea02d9f5058 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:10.778055 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae8903b-3b57-4d43-8350-b898539da3ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b65b9e1603349d2880eb2b4d793211e, user_id=aa9c3da38c7e44c2892b117c204ca7bc, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:10.779253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-eae8903b-3b57-4d43-8350-b898539da3ed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 12:33:10.779253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jun 29 12:33:10.779253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: Jun 29 12:33:10.781220 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae8903b-3b57-4d43-8350-b898539da3ed admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:10.783851 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 788/1574] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:10 2022] PUT /identity/v3/projects/0ed882ae485441db99ccf6109be51b22/users/6252d0519ed9421e89710ca5294524c5/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:10.784670 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349821e-52e3-4b27-a836-6ea02d9f5058 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:10.799165 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 787/1575] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:10 2022] PUT /identity/v3/projects/1b65b9e1603349d2880eb2b4d793211e/users/aa9c3da38c7e44c2892b117c204ca7bc/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:10.799995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae8903b-3b57-4d43-8350-b898539da3ed admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:10.860643 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae8903b-3b57-4d43-8350-b898539da3ed admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:10.861287 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349821e-52e3-4b27-a836-6ea02d9f5058 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:10.865578 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b44f33-6a5e-4546-8424-db361c9a0373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:10.865762 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd4981-d32f-42c6-a4c4-3c639e5a434d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:10.866250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b44f33-6a5e-4546-8424-db361c9a0373 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.866513 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd4981-d32f-42c6-a4c4-3c639e5a434d admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.866636 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b44f33-6a5e-4546-8424-db361c9a0373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.866835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd4981-d32f-42c6-a4c4-3c639e5a434d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.866929 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b44f33-6a5e-4546-8424-db361c9a0373 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.867078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd4981-d32f-42c6-a4c4-3c639e5a434d admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.867523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b44f33-6a5e-4546-8424-db361c9a0373 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:10.867773 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd4981-d32f-42c6-a4c4-3c639e5a434d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:10.869431 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b44f33-6a5e-4546-8424-db361c9a0373 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:10.869666 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd4981-d32f-42c6-a4c4-3c639e5a434d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:10.894786 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 789/1576] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:10 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:10.896479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b44f33-6a5e-4546-8424-db361c9a0373 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:10.900049 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 788/1577] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:10 2022] POST /identity/v3/projects => generated 403 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:10.900049 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd4981-d32f-42c6-a4c4-3c639e5a434d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:10.947235 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:10.948794 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:10.964146 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd4981-d32f-42c6-a4c4-3c639e5a434d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:10.964710 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b44f33-6a5e-4546-8424-db361c9a0373 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:10.968621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56e889-85e6-404c-b891-76b1e6b3745d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:10.969271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56e889-85e6-404c-b891-76b1e6b3745d admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.969402 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb5007-44f1-44c0-8c8b-c26a6cf091a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:10.969555 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56e889-85e6-404c-b891-76b1e6b3745d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.969831 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56e889-85e6-404c-b891-76b1e6b3745d admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.970021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb5007-44f1-44c0-8c8b-c26a6cf091a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:10.970312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb5007-44f1-44c0-8c8b-c26a6cf091a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:10.970616 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c56e889-85e6-404c-b891-76b1e6b3745d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:10.970717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb5007-44f1-44c0-8c8b-c26a6cf091a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:10.971304 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb5007-44f1-44c0-8c8b-c26a6cf091a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:10.972527 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c56e889-85e6-404c-b891-76b1e6b3745d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:10.973355 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb5007-44f1-44c0-8c8b-c26a6cf091a7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:10.999203 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 789/1578] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:10 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:11.000015 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56e889-85e6-404c-b891-76b1e6b3745d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:11.003940 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 790/1579] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:10 2022] POST /identity/v3/projects => generated 391 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:11.004799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb5007-44f1-44c0-8c8b-c26a6cf091a7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:11.066091 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56e889-85e6-404c-b891-76b1e6b3745d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:11.069118 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb5007-44f1-44c0-8c8b-c26a6cf091a7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:11.070933 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc586f6-ba13-4771-96ab-6a72237752a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:11.071608 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc586f6-ba13-4771-96ab-6a72237752a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.071898 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc586f6-ba13-4771-96ab-6a72237752a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.072179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc586f6-ba13-4771-96ab-6a72237752a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.072904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc586f6-ba13-4771-96ab-6a72237752a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:11.073811 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824cea15-6295-450b-96c5-e8b8312e254e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:11.074637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cea15-6295-450b-96c5-e8b8312e254e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.074927 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cea15-6295-450b-96c5-e8b8312e254e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.075208 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc586f6-ba13-4771-96ab-6a72237752a8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:11.075325 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cea15-6295-450b-96c5-e8b8312e254e admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.075897 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824cea15-6295-450b-96c5-e8b8312e254e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:11.077836 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824cea15-6295-450b-96c5-e8b8312e254e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:11.099837 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 791/1580] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:11 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:11.103316 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 790/1581] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:10 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:11.103482 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11367934-4413-4250-9388-bd7989ba728b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.103482 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11367934-4413-4250-9388-bd7989ba728b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.103628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa1d5b-85bf-4b1f-9af6-98485b990287 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.103702 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11367934-4413-4250-9388-bd7989ba728b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.103774 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa1d5b-85bf-4b1f-9af6-98485b990287 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.103774 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa1d5b-85bf-4b1f-9af6-98485b990287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.121743 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-11367934-4413-4250-9388-bd7989ba728b None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:11.122890 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-11367934-4413-4250-9388-bd7989ba728b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:11.131039 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 792/1582] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:11.131833 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11367934-4413-4250-9388-bd7989ba728b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:11.132846 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f1aa1d5b-85bf-4b1f-9af6-98485b990287 None None] MFA Rules not processed for user `aa9c3da38c7e44c2892b117c204ca7bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:11.134247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f1aa1d5b-85bf-4b1f-9af6-98485b990287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:11.142838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11367934-4413-4250-9388-bd7989ba728b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:11.148431 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a998e-3e5c-4ed0-9910-5aa3878373d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:11.149106 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a998e-3e5c-4ed0-9910-5aa3878373d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.149387 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a998e-3e5c-4ed0-9910-5aa3878373d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.149663 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831a998e-3e5c-4ed0-9910-5aa3878373d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.150356 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831a998e-3e5c-4ed0-9910-5aa3878373d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:11.152570 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831a998e-3e5c-4ed0-9910-5aa3878373d2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:11.175648 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 793/1583] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:11 2022] POST /identity/v3/users => generated 302 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:11.179453 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a998e-3e5c-4ed0-9910-5aa3878373d2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:11.188987 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831a998e-3e5c-4ed0-9910-5aa3878373d2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:11.193802 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87392801-66f4-4699-9757-0bf4e054b53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:11.194523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87392801-66f4-4699-9757-0bf4e054b53a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.194813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87392801-66f4-4699-9757-0bf4e054b53a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.195152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87392801-66f4-4699-9757-0bf4e054b53a admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.195852 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87392801-66f4-4699-9757-0bf4e054b53a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:11.197756 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87392801-66f4-4699-9757-0bf4e054b53a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:11.203288 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 791/1584] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:11 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:11.204105 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa1d5b-85bf-4b1f-9af6-98485b990287 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:11.215542 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa1d5b-85bf-4b1f-9af6-98485b990287 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:11.221122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 794/1585] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:11 2022] POST /identity/v3/users => generated 288 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:11.221122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87392801-66f4-4699-9757-0bf4e054b53a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:11.221337 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c1eb5-a9be-4f14-9480-79726d9ff79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:11.221337 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c1eb5-a9be-4f14-9480-79726d9ff79d admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.221337 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c1eb5-a9be-4f14-9480-79726d9ff79d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.221337 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c1eb5-a9be-4f14-9480-79726d9ff79d admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.221976 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817c1eb5-a9be-4f14-9480-79726d9ff79d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:11.224717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817c1eb5-a9be-4f14-9480-79726d9ff79d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:11.229398 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87392801-66f4-4699-9757-0bf4e054b53a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:11.234123 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b140284-9549-4b2a-bf20-6c41261fd3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:11.234855 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b140284-9549-4b2a-bf20-6c41261fd3e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.235220 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b140284-9549-4b2a-bf20-6c41261fd3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.235362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b140284-9549-4b2a-bf20-6c41261fd3e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.236168 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b140284-9549-4b2a-bf20-6c41261fd3e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:11.238350 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b140284-9549-4b2a-bf20-6c41261fd3e1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:11.245769 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 792/1586] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:11 2022] POST /identity/v3/users => generated 290 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:11.245769 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c1eb5-a9be-4f14-9480-79726d9ff79d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:11.257728 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c1eb5-a9be-4f14-9480-79726d9ff79d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:11.262876 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0708f7-b5ed-4d96-91e7-774d3f543f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:11.263534 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0708f7-b5ed-4d96-91e7-774d3f543f31 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.263823 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0708f7-b5ed-4d96-91e7-774d3f543f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.264103 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0708f7-b5ed-4d96-91e7-774d3f543f31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.267705 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 795/1587] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:11 2022] POST /identity/v3/users => generated 296 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:11.267866 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0708f7-b5ed-4d96-91e7-774d3f543f31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:11.267967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b140284-9549-4b2a-bf20-6c41261fd3e1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:11.268063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0708f7-b5ed-4d96-91e7-774d3f543f31 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:11.276940 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b140284-9549-4b2a-bf20-6c41261fd3e1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:11.277863 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 793/1588] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:11.278617 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0708f7-b5ed-4d96-91e7-774d3f543f31 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:11.283900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf0b63-8e7e-460e-a151-abdb47aa983b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:11.284570 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf0b63-8e7e-460e-a151-abdb47aa983b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.284858 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf0b63-8e7e-460e-a151-abdb47aa983b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.285149 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acf0b63-8e7e-460e-a151-abdb47aa983b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.285879 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acf0b63-8e7e-460e-a151-abdb47aa983b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:11.288171 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acf0b63-8e7e-460e-a151-abdb47aa983b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:11.303190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 796/1589] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:11.303190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf0b63-8e7e-460e-a151-abdb47aa983b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:11.309393 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:11.311288 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acf0b63-8e7e-460e-a151-abdb47aa983b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:11.316250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22369e-7c5d-4a69-b55c-954c5ca172f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:11.316897 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22369e-7c5d-4a69-b55c-954c5ca172f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.318628 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22369e-7c5d-4a69-b55c-954c5ca172f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.319101 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b22369e-7c5d-4a69-b55c-954c5ca172f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.320000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b22369e-7c5d-4a69-b55c-954c5ca172f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:11.321736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b22369e-7c5d-4a69-b55c-954c5ca172f8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:11.331804 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 797/1590] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:11.332582 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22369e-7c5d-4a69-b55c-954c5ca172f8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:11.343194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b22369e-7c5d-4a69-b55c-954c5ca172f8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:11.346658 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0708f7-b5ed-4d96-91e7-774d3f543f31 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:11.349412 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7121730-2a2f-43fe-994d-91093d62384c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:11.350086 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7121730-2a2f-43fe-994d-91093d62384c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.350349 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7121730-2a2f-43fe-994d-91093d62384c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.350489 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54080a-6905-4d2c-ab42-e4571daea4c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:11.350700 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7121730-2a2f-43fe-994d-91093d62384c admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.351257 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54080a-6905-4d2c-ab42-e4571daea4c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.351500 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7121730-2a2f-43fe-994d-91093d62384c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:11.351616 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54080a-6905-4d2c-ab42-e4571daea4c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.352045 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f54080a-6905-4d2c-ab42-e4571daea4c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.353560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7121730-2a2f-43fe-994d-91093d62384c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:11.361671 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54080a-6905-4d2c-ab42-e4571daea4c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:11.363282 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f54080a-6905-4d2c-ab42-e4571daea4c8 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:11.373647 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 794/1591] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:11.374465 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54080a-6905-4d2c-ab42-e4571daea4c8 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:11.380771 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 798/1592] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:11 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:11.381648 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7121730-2a2f-43fe-994d-91093d62384c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:11.387444 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f54080a-6905-4d2c-ab42-e4571daea4c8 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:11.403423 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:INSERT=14 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:11.460002 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1da71b-be05-442e-b801-def515cf01bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:11.460681 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1da71b-be05-442e-b801-def515cf01bc admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.460968 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1da71b-be05-442e-b801-def515cf01bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.461487 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1da71b-be05-442e-b801-def515cf01bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.461941 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7121730-2a2f-43fe-994d-91093d62384c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:11.462454 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1da71b-be05-442e-b801-def515cf01bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:11.464265 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1da71b-be05-442e-b801-def515cf01bc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:11.466994 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d725f-1a3d-4b42-a023-37c182a7d6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:11.467676 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d725f-1a3d-4b42-a023-37c182a7d6ad admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.467961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d725f-1a3d-4b42-a023-37c182a7d6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.468238 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d725f-1a3d-4b42-a023-37c182a7d6ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.468972 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d725f-1a3d-4b42-a023-37c182a7d6ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:11.470736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d725f-1a3d-4b42-a023-37c182a7d6ad admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:11.484866 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 799/1593] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:11.484866 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d725f-1a3d-4b42-a023-37c182a7d6ad admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:11.485175 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 795/1594] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:11.485175 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1da71b-be05-442e-b801-def515cf01bc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:11.492869 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d725f-1a3d-4b42-a023-37c182a7d6ad admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:11.495750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1da71b-be05-442e-b801-def515cf01bc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:11.498207 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0592bfeb-8930-43f5-b9b3-2d1b72459a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:11.498559 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592bfeb-8930-43f5-b9b3-2d1b72459a0f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.498732 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592bfeb-8930-43f5-b9b3-2d1b72459a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.499052 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592bfeb-8930-43f5-b9b3-2d1b72459a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.499924 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0592bfeb-8930-43f5-b9b3-2d1b72459a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:11.500966 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d81525-7bac-426c-971d-67fce872787f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:11.501541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0592bfeb-8930-43f5-b9b3-2d1b72459a0f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:11.501662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d81525-7bac-426c-971d-67fce872787f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.501929 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d81525-7bac-426c-971d-67fce872787f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.502211 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d81525-7bac-426c-971d-67fce872787f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.502967 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d81525-7bac-426c-971d-67fce872787f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:11.504692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d81525-7bac-426c-971d-67fce872787f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:11.514902 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 800/1595] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:11.516029 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0592bfeb-8930-43f5-b9b3-2d1b72459a0f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:11.516397 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 796/1596] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:11.517222 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d81525-7bac-426c-971d-67fce872787f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:11.530846 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d81525-7bac-426c-971d-67fce872787f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:11.531084 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0592bfeb-8930-43f5-b9b3-2d1b72459a0f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:11.535606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67040e0f-7ac5-4f3a-aac0-f348a9b497b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:11.536327 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67040e0f-7ac5-4f3a-aac0-f348a9b497b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.536616 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67040e0f-7ac5-4f3a-aac0-f348a9b497b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.536892 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67040e0f-7ac5-4f3a-aac0-f348a9b497b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.537755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67040e0f-7ac5-4f3a-aac0-f348a9b497b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:11.538463 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9e6d0-c0ee-45bb-ab68-f7dbdd51798e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:11.539511 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67040e0f-7ac5-4f3a-aac0-f348a9b497b6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:11.539942 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9e6d0-c0ee-45bb-ab68-f7dbdd51798e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.540494 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9e6d0-c0ee-45bb-ab68-f7dbdd51798e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.541302 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9e6d0-c0ee-45bb-ab68-f7dbdd51798e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.542184 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9e6d0-c0ee-45bb-ab68-f7dbdd51798e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:11.544576 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9e6d0-c0ee-45bb-ab68-f7dbdd51798e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:11.556173 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 797/1597] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:11.556173 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67040e0f-7ac5-4f3a-aac0-f348a9b497b6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:11.563795 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 801/1598] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:11.563795 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9e6d0-c0ee-45bb-ab68-f7dbdd51798e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:11.568731 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67040e0f-7ac5-4f3a-aac0-f348a9b497b6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:11.577970 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4e4d3-8186-49aa-92e1-1dc785ce9207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:11.578860 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4e4d3-8186-49aa-92e1-1dc785ce9207 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.579149 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4e4d3-8186-49aa-92e1-1dc785ce9207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.579445 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4e4d3-8186-49aa-92e1-1dc785ce9207 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.580646 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca4e4d3-8186-49aa-92e1-1dc785ce9207 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:11.581373 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9e6d0-c0ee-45bb-ab68-f7dbdd51798e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:11.583502 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca4e4d3-8186-49aa-92e1-1dc785ce9207 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:11.587328 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66c7fd-7317-4180-a4a9-802f0abe3e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:11.588060 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66c7fd-7317-4180-a4a9-802f0abe3e40 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.588350 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66c7fd-7317-4180-a4a9-802f0abe3e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.588627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66c7fd-7317-4180-a4a9-802f0abe3e40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.589349 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d66c7fd-7317-4180-a4a9-802f0abe3e40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:11.591153 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d66c7fd-7317-4180-a4a9-802f0abe3e40 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:11.604334 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 802/1599] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:11.605183 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66c7fd-7317-4180-a4a9-802f0abe3e40 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:11.605312 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 798/1600] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:11 2022] POST /identity/v3/users => generated 288 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:11.606061 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4e4d3-8186-49aa-92e1-1dc785ce9207 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:11.620385 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca4e4d3-8186-49aa-92e1-1dc785ce9207 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:11.621892 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66c7fd-7317-4180-a4a9-802f0abe3e40 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:11.625745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac5f02c-dc1a-4fe1-8519-8724a966ee8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:11.626452 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5f02c-dc1a-4fe1-8519-8724a966ee8a admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.626742 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5f02c-dc1a-4fe1-8519-8724a966ee8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.627029 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5f02c-dc1a-4fe1-8519-8724a966ee8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.627156 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52f924-4a35-4143-bc65-d99a0ebc8089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:11.627771 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac5f02c-dc1a-4fe1-8519-8724a966ee8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:11.627898 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52f924-4a35-4143-bc65-d99a0ebc8089 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.628138 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52f924-4a35-4143-bc65-d99a0ebc8089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.628436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52f924-4a35-4143-bc65-d99a0ebc8089 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.629251 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e52f924-4a35-4143-bc65-d99a0ebc8089 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:11.629360 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac5f02c-dc1a-4fe1-8519-8724a966ee8a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:11.631048 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e52f924-4a35-4143-bc65-d99a0ebc8089 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:11.641409 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 799/1601] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:11.648251 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 803/1602] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:11.648412 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac5f02c-dc1a-4fe1-8519-8724a966ee8a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:11.648499 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52f924-4a35-4143-bc65-d99a0ebc8089 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:11.654258 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac5f02c-dc1a-4fe1-8519-8724a966ee8a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:11.654843 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52f924-4a35-4143-bc65-d99a0ebc8089 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:11.658968 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1658de2-21a7-47b5-b623-a09f93f7b0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:11.659953 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d55e7-a9e1-416f-8d7d-776f40c06a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:11.660124 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1658de2-21a7-47b5-b623-a09f93f7b0a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.660418 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1658de2-21a7-47b5-b623-a09f93f7b0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.660703 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1658de2-21a7-47b5-b623-a09f93f7b0a0 admin admin] PATH_INFO: `/v3/projects/c8efd9db53c14bf69dc0ebd8ebac75fc/users/08d02203c5654d71853045704fb0d914/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.660900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d55e7-a9e1-416f-8d7d-776f40c06a44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.661376 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d55e7-a9e1-416f-8d7d-776f40c06a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.661661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d55e7-a9e1-416f-8d7d-776f40c06a44 admin admin] PATH_INFO: `/v3/projects/0ecbd8a904ce455bb6d0a71db1e6edac/users/558836b50d72443d97d97597b5574954/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.677513 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1658de2-21a7-47b5-b623-a09f93f7b0a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8efd9db53c14bf69dc0ebd8ebac75fc, user_id=08d02203c5654d71853045704fb0d914, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:11.680195 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347d55e7-a9e1-416f-8d7d-776f40c06a44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ecbd8a904ce455bb6d0a71db1e6edac, user_id=558836b50d72443d97d97597b5574954, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:11.680361 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1658de2-21a7-47b5-b623-a09f93f7b0a0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:11.683228 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347d55e7-a9e1-416f-8d7d-776f40c06a44 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:11.695849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 800/1603] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:11 2022] PUT /identity/v3/projects/c8efd9db53c14bf69dc0ebd8ebac75fc/users/08d02203c5654d71853045704fb0d914/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:11.696652 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1658de2-21a7-47b5-b623-a09f93f7b0a0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:11.699795 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 804/1604] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:11 2022] PUT /identity/v3/projects/0ecbd8a904ce455bb6d0a71db1e6edac/users/558836b50d72443d97d97597b5574954/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:11.701018 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d55e7-a9e1-416f-8d7d-776f40c06a44 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:11.759151 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:11.781062 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1658de2-21a7-47b5-b623-a09f93f7b0a0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:11.784459 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d55e7-a9e1-416f-8d7d-776f40c06a44 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:11.787450 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bafb6ab-7a77-47a6-a4b9-eb5fb469a912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:11.788757 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bafb6ab-7a77-47a6-a4b9-eb5fb469a912 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.788757 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bafb6ab-7a77-47a6-a4b9-eb5fb469a912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.788994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bafb6ab-7a77-47a6-a4b9-eb5fb469a912 admin admin] PATH_INFO: `/v3/projects/6fd787ac66684a5b953f94e1273a75e4/users/39886efe36ce4ac9b55f06dffd54d1a3/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.791199 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0b307-78fa-4e39-920d-ed8092370d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:11.792446 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0b307-78fa-4e39-920d-ed8092370d8c admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.792740 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0b307-78fa-4e39-920d-ed8092370d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.793063 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0b307-78fa-4e39-920d-ed8092370d8c admin admin] PATH_INFO: `/v3/projects/0f625199b4eb48c98944ea79a90f2841/users/a4a30b65a91a4fd89e8b59df670f60ad/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.806768 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bafb6ab-7a77-47a6-a4b9-eb5fb469a912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fd787ac66684a5b953f94e1273a75e4, user_id=39886efe36ce4ac9b55f06dffd54d1a3, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:11.809546 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bafb6ab-7a77-47a6-a4b9-eb5fb469a912 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:11.812638 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca0b307-78fa-4e39-920d-ed8092370d8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f625199b4eb48c98944ea79a90f2841, user_id=a4a30b65a91a4fd89e8b59df670f60ad, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:11.816313 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca0b307-78fa-4e39-920d-ed8092370d8c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:11.827759 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 801/1605] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:11 2022] PUT /identity/v3/projects/6fd787ac66684a5b953f94e1273a75e4/users/39886efe36ce4ac9b55f06dffd54d1a3/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:11.828813 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bafb6ab-7a77-47a6-a4b9-eb5fb469a912 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:11.844019 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 805/1606] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:11 2022] PUT /identity/v3/projects/0f625199b4eb48c98944ea79a90f2841/users/a4a30b65a91a4fd89e8b59df670f60ad/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:11.844646 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0b307-78fa-4e39-920d-ed8092370d8c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:11.893377 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:11.916513 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bafb6ab-7a77-47a6-a4b9-eb5fb469a912 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:11.939314 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0b307-78fa-4e39-920d-ed8092370d8c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:11.946137 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d7d947-683d-4f5e-a7b6-b31cb821e8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:11.947303 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7d947-683d-4f5e-a7b6-b31cb821e8bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.947672 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7d947-683d-4f5e-a7b6-b31cb821e8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.948019 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d7d947-683d-4f5e-a7b6-b31cb821e8bc admin admin] PATH_INFO: `/v3/projects/8be6d4a4e2344e25b7661d69a9e222c8/users/350a57ee307a4419974dcecf37e466e5/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.963854 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d7d947-683d-4f5e-a7b6-b31cb821e8bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8be6d4a4e2344e25b7661d69a9e222c8, user_id=350a57ee307a4419974dcecf37e466e5, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:11.966926 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d7d947-683d-4f5e-a7b6-b31cb821e8bc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:11.986702 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 806/1607] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:11 2022] PUT /identity/v3/projects/8be6d4a4e2344e25b7661d69a9e222c8/users/350a57ee307a4419974dcecf37e466e5/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:11.986702 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d7d947-683d-4f5e-a7b6-b31cb821e8bc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:11.987440 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc34f90-2242-49e1-915f-c14759687d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:11.988206 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc34f90-2242-49e1-915f-c14759687d63 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:11.988664 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc34f90-2242-49e1-915f-c14759687d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:11.988950 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc34f90-2242-49e1-915f-c14759687d63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:11.990241 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc34f90-2242-49e1-915f-c14759687d63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:11.991754 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc34f90-2242-49e1-915f-c14759687d63 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:12.006703 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 802/1608] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:12.008066 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97704447-9008-443c-803a-b7c10c25e286 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:12.008359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97704447-9008-443c-803a-b7c10c25e286 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:12.008639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97704447-9008-443c-803a-b7c10c25e286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:12.028935 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-97704447-9008-443c-803a-b7c10c25e286 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:12.030356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-97704447-9008-443c-803a-b7c10c25e286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:12.069525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d7d947-683d-4f5e-a7b6-b31cb821e8bc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:12.074522 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9be5cd0-78c3-44a1-b789-f7a49535fc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:12.077689 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be5cd0-78c3-44a1-b789-f7a49535fc91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:12.077689 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be5cd0-78c3-44a1-b789-f7a49535fc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:12.077689 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be5cd0-78c3-44a1-b789-f7a49535fc91 admin admin] PATH_INFO: `/v3/projects/46f5397c062a4868a0282f7535f84f7a/users/8a232c3292ec4a0abda3a8bc91c5a267/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:12.099451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9be5cd0-78c3-44a1-b789-f7a49535fc91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46f5397c062a4868a0282f7535f84f7a, user_id=8a232c3292ec4a0abda3a8bc91c5a267, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:12.102599 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9be5cd0-78c3-44a1-b789-f7a49535fc91 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:12.106822 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 803/1609] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:12.107405 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86879e48-8307-48d4-b243-21d17437b92e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:12.107777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86879e48-8307-48d4-b243-21d17437b92e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:12.108142 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86879e48-8307-48d4-b243-21d17437b92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:12.123584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 807/1610] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:11 2022] PUT /identity/v3/projects/46f5397c062a4868a0282f7535f84f7a/users/8a232c3292ec4a0abda3a8bc91c5a267/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:12.124963 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927792f-89c3-40ba-b42b-e54d08039689 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:12.125289 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927792f-89c3-40ba-b42b-e54d08039689 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:12.125614 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927792f-89c3-40ba-b42b-e54d08039689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:12.142485 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:12.148949 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-86879e48-8307-48d4-b243-21d17437b92e None None] MFA Rules not processed for user `08d02203c5654d71853045704fb0d914`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:12.150039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-86879e48-8307-48d4-b243-21d17437b92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:12.159704 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8927792f-89c3-40ba-b42b-e54d08039689 None None] MFA Rules not processed for user `39886efe36ce4ac9b55f06dffd54d1a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:12.162078 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8927792f-89c3-40ba-b42b-e54d08039689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:12.251755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 808/1611] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:12.251755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8eac80-8cd5-41fc-9926-fd61c606c9a8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:12.251755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8eac80-8cd5-41fc-9926-fd61c606c9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:12.251755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8eac80-8cd5-41fc-9926-fd61c606c9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:12.267521 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 804/1612] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:12.268474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5ba2d-cb91-45c0-9420-d9ddc66487b1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:12.268758 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5ba2d-cb91-45c0-9420-d9ddc66487b1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:12.269109 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5ba2d-cb91-45c0-9420-d9ddc66487b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:12.293845 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-aec5ba2d-cb91-45c0-9420-d9ddc66487b1 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:12.295732 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-aec5ba2d-cb91-45c0-9420-d9ddc66487b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:12.309551 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1c8eac80-8cd5-41fc-9926-fd61c606c9a8 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:12.313196 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1c8eac80-8cd5-41fc-9926-fd61c606c9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:12.350711 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:12.404193 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 805/1613] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:12.405415 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5ba2d-cb91-45c0-9420-d9ddc66487b1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:12.410632 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 809/1614] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:12.412057 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8eac80-8cd5-41fc-9926-fd61c606c9a8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:12.431531 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5ba2d-cb91-45c0-9420-d9ddc66487b1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:12.437489 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ef8612-4c96-4aa3-bc16-ffec2764fbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:12.439926 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef8612-4c96-4aa3-bc16-ffec2764fbc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:12.440427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef8612-4c96-4aa3-bc16-ffec2764fbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:12.441013 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef8612-4c96-4aa3-bc16-ffec2764fbc6 admin admin] PATH_INFO: `/v3/projects/587b93767a5645c99b26b4acd3600515/users/bb55a30c00f24cb995c08c9d057cc567/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:12.461412 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef8612-4c96-4aa3-bc16-ffec2764fbc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=587b93767a5645c99b26b4acd3600515, user_id=bb55a30c00f24cb995c08c9d057cc567, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:12.464322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ef8612-4c96-4aa3-bc16-ffec2764fbc6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:12.482392 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8eac80-8cd5-41fc-9926-fd61c606c9a8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:12.485621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 806/1615] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:12 2022] PUT /identity/v3/projects/587b93767a5645c99b26b4acd3600515/users/bb55a30c00f24cb995c08c9d057cc567/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:12.487103 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad674f7b-98df-438b-ab23-bcd8e8ddbb5e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:12.487398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad674f7b-98df-438b-ab23-bcd8e8ddbb5e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:12.487684 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad674f7b-98df-438b-ab23-bcd8e8ddbb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:12.511913 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ad674f7b-98df-438b-ab23-bcd8e8ddbb5e None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:12.513261 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ad674f7b-98df-438b-ab23-bcd8e8ddbb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:12.564988 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df46f3-3113-4c5e-b668-7991fa787f21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:12.565653 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df46f3-3113-4c5e-b668-7991fa787f21 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:12.565987 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df46f3-3113-4c5e-b668-7991fa787f21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:12.566272 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df46f3-3113-4c5e-b668-7991fa787f21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:12.618850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 807/1616] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:12.620992 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e1abf-4135-4e71-ab31-63d91f3154ae None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:12.621361 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e1abf-4135-4e71-ab31-63d91f3154ae None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:12.621758 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e1abf-4135-4e71-ab31-63d91f3154ae None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:12.624144 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 808/1617] 10.222.0.52 () {54 vars in 961 bytes} [Wed Jun 29 12:33:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:33:12.625374 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b405317-b3f0-4183-9d10-904c04ff1598 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:12.625791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b405317-b3f0-4183-9d10-904c04ff1598 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:12.626147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b405317-b3f0-4183-9d10-904c04ff1598 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:12.635810 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 809/1618] 10.222.0.52 () {54 vars in 961 bytes} [Wed Jun 29 12:33:12 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:33:12.637206 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b405317-b3f0-4183-9d10-904c04ff1598 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:12.640971 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2b405317-b3f0-4183-9d10-904c04ff1598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:12.659082 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b405317-b3f0-4183-9d10-904c04ff1598 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:12.663828 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52df46f3-3113-4c5e-b668-7991fa787f21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:12.665084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a0b6c-2f94-4496-9a6f-a0e1f74bc2c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:12.665381 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52df46f3-3113-4c5e-b668-7991fa787f21 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:12.665865 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a0b6c-2f94-4496-9a6f-a0e1f74bc2c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:12.666219 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a0b6c-2f94-4496-9a6f-a0e1f74bc2c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:12.666605 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a0b6c-2f94-4496-9a6f-a0e1f74bc2c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:12.676141 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 810/1619] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:12.677237 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df46f3-3113-4c5e-b668-7991fa787f21 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:12.681025 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a0b6c-2f94-4496-9a6f-a0e1f74bc2c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:12.683169 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a0b6c-2f94-4496-9a6f-a0e1f74bc2c3 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:12.685390 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df46f3-3113-4c5e-b668-7991fa787f21 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:12.689126 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed111c88-9187-4073-88bf-95d1d1de9705 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:12.689777 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed111c88-9187-4073-88bf-95d1d1de9705 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:12.690067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed111c88-9187-4073-88bf-95d1d1de9705 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:12.690348 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed111c88-9187-4073-88bf-95d1d1de9705 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:12.697445 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 810/1620] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:12.698538 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a0b6c-2f94-4496-9a6f-a0e1f74bc2c3 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:12.715663 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a0b6c-2f94-4496-9a6f-a0e1f74bc2c3 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:12.718006 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed111c88-9187-4073-88bf-95d1d1de9705 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:12.719490 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed111c88-9187-4073-88bf-95d1d1de9705 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:12.723439 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c256ae8-13d4-4f33-b8e2-d9e216cd983d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:12.725440 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c256ae8-13d4-4f33-b8e2-d9e216cd983d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:12.725875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c256ae8-13d4-4f33-b8e2-d9e216cd983d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:12.726163 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c256ae8-13d4-4f33-b8e2-d9e216cd983d admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:12.727053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c256ae8-13d4-4f33-b8e2-d9e216cd983d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:12.729717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c256ae8-13d4-4f33-b8e2-d9e216cd983d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:12.731846 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:12.742614 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 811/1621] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:12.745523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24160e9-5971-4b8c-bef9-fa38e7d001ba None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:12.745823 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24160e9-5971-4b8c-bef9-fa38e7d001ba None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:12.746110 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24160e9-5971-4b8c-bef9-fa38e7d001ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:12.751885 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 811/1622] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:12.751885 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c256ae8-13d4-4f33-b8e2-d9e216cd983d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:12.769199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c256ae8-13d4-4f33-b8e2-d9e216cd983d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:12.778953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e70d1f3-89f8-40f0-809b-9dcf8c0145b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:12.779635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70d1f3-89f8-40f0-809b-9dcf8c0145b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:12.779924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70d1f3-89f8-40f0-809b-9dcf8c0145b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:12.780210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e70d1f3-89f8-40f0-809b-9dcf8c0145b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:12.782538 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e24160e9-5971-4b8c-bef9-fa38e7d001ba None None] MFA Rules not processed for user `3ac7391a3c41499d94e5f97478797313`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:12.784219 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e24160e9-5971-4b8c-bef9-fa38e7d001ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:12.793202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70d1f3-89f8-40f0-809b-9dcf8c0145b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:12.795033 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e70d1f3-89f8-40f0-809b-9dcf8c0145b3 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:12.804516 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:12.809799 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 812/1623] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:12.811247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b748e5d-3740-42a3-88e8-aa300d54543a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:12.811549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b748e5d-3740-42a3-88e8-aa300d54543a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:12.811884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b748e5d-3740-42a3-88e8-aa300d54543a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:12.840932 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7b748e5d-3740-42a3-88e8-aa300d54543a None None] MFA Rules not processed for user `3ac7391a3c41499d94e5f97478797313`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:12.843568 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7b748e5d-3740-42a3-88e8-aa300d54543a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:12.894087 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 812/1624] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Jun 29 12:33:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:12.895780 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24160e9-5971-4b8c-bef9-fa38e7d001ba None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:12.910793 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24160e9-5971-4b8c-bef9-fa38e7d001ba None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:12.923236 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0d10a-6a8b-4b15-ab53-0ddf13991917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:12.924810 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0d10a-6a8b-4b15-ab53-0ddf13991917 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:12.925140 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0d10a-6a8b-4b15-ab53-0ddf13991917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:12.925449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea0d10a-6a8b-4b15-ab53-0ddf13991917 admin admin] PATH_INFO: `/v3/projects/587b93767a5645c99b26b4acd3600515/users/bb55a30c00f24cb995c08c9d057cc567/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:12.929471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0d10a-6a8b-4b15-ab53-0ddf13991917 admin admin] RBAC: Authorizing `identity:create_grant(project_id=587b93767a5645c99b26b4acd3600515, user_id=bb55a30c00f24cb995c08c9d057cc567, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:12.932920 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea0d10a-6a8b-4b15-ab53-0ddf13991917 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:12.949986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 813/1625] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:12 2022] PUT /identity/v3/projects/587b93767a5645c99b26b4acd3600515/users/bb55a30c00f24cb995c08c9d057cc567/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:12.951081 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0d10a-6a8b-4b15-ab53-0ddf13991917 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:12.982193 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 813/1626] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Jun 29 12:33:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:12.984081 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70abda-bd23-453b-9014-8c19be930032 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:12.984764 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70abda-bd23-453b-9014-8c19be930032 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:12.985406 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70abda-bd23-453b-9014-8c19be930032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:13.043179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-be70abda-bd23-453b-9014-8c19be930032 None None] MFA Rules not processed for user `bb55a30c00f24cb995c08c9d057cc567`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:13.048805 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-be70abda-bd23-453b-9014-8c19be930032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:13.085754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea0d10a-6a8b-4b15-ab53-0ddf13991917 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:13.094507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef28a9-8dae-4ab0-9e91-bd737102c07f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:13.095620 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef28a9-8dae-4ab0-9e91-bd737102c07f service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:13.096298 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef28a9-8dae-4ab0-9e91-bd737102c07f service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:13.096791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef28a9-8dae-4ab0-9e91-bd737102c07f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:13.199761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 814/1627] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:12 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:13.200713 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be70abda-bd23-453b-9014-8c19be930032 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:13.206454 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef28a9-8dae-4ab0-9e91-bd737102c07f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:13.210192 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef28a9-8dae-4ab0-9e91-bd737102c07f service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:13.224639 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 814/1628] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:13.225564 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef28a9-8dae-4ab0-9e91-bd737102c07f service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:13.228944 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be70abda-bd23-453b-9014-8c19be930032 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:13.244301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a8fe9-6a3c-4031-a03a-4f02046494b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:13.244845 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a8fe9-6a3c-4031-a03a-4f02046494b4 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:13.245076 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a8fe9-6a3c-4031-a03a-4f02046494b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:13.245356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a8fe9-6a3c-4031-a03a-4f02046494b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:13.280667 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:13.317046 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef28a9-8dae-4ab0-9e91-bd737102c07f service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:13.323522 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f8083-42eb-4989-80d3-8821bdb686f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:13.324200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f8083-42eb-4989-80d3-8821bdb686f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:13.324649 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f8083-42eb-4989-80d3-8821bdb686f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:13.324948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f8083-42eb-4989-80d3-8821bdb686f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:13.350344 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f8083-42eb-4989-80d3-8821bdb686f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:13.352118 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f8083-42eb-4989-80d3-8821bdb686f0 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:13.377450 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 815/1629] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:13 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:13.379601 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a8fe9-6a3c-4031-a03a-4f02046494b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:13.384028 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a8fe9-6a3c-4031-a03a-4f02046494b4 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:13.389433 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:13.400523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 815/1630] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:13.591733 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f8083-42eb-4989-80d3-8821bdb686f0 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:13.605597 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f8083-42eb-4989-80d3-8821bdb686f0 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:13.618013 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79957f5-988c-4663-a5a8-4ce72a29783f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:13.619199 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79957f5-988c-4663-a5a8-4ce72a29783f service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:13.619798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79957f5-988c-4663-a5a8-4ce72a29783f service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:13.622839 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79957f5-988c-4663-a5a8-4ce72a29783f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:13.847602 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79957f5-988c-4663-a5a8-4ce72a29783f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:13.847602 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79957f5-988c-4663-a5a8-4ce72a29783f service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:13.847602 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 816/1631] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:13 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:13.862569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b2ea3-baf1-4890-8881-1ba5b0b849d7 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:13.862927 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b2ea3-baf1-4890-8881-1ba5b0b849d7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:13.863293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b2ea3-baf1-4890-8881-1ba5b0b849d7 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:13.865462 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 816/1632] 10.222.0.52 () {54 vars in 912 bytes} [Wed Jun 29 12:33:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:33:13.872349 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b0651-26c7-4402-b622-f821cb63fedd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:13.872721 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b0651-26c7-4402-b622-f821cb63fedd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:13.873063 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b0651-26c7-4402-b622-f821cb63fedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:13.901245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cb3b0651-26c7-4402-b622-f821cb63fedd None None] MFA Rules not processed for user `3ac7391a3c41499d94e5f97478797313`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:13.902610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cb3b0651-26c7-4402-b622-f821cb63fedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:13.938031 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 817/1633] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jun 29 12:33:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:33:13.946174 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b2ea3-baf1-4890-8881-1ba5b0b849d7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:13.957841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282beb6-b7e5-4dc3-aad4-55323b7edec6 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:13.959134 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282beb6-b7e5-4dc3-aad4-55323b7edec6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:13.959411 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282beb6-b7e5-4dc3-aad4-55323b7edec6 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:13.961033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 818/1634] 10.222.0.52 () {54 vars in 912 bytes} [Wed Jun 29 12:33:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:33:13.965248 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b2ea3-baf1-4890-8881-1ba5b0b849d7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:13.967570 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f220c9-0ba2-4604-8f8c-fdd9a59183bd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:13.967926 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f220c9-0ba2-4604-8f8c-fdd9a59183bd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:13.968276 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f220c9-0ba2-4604-8f8c-fdd9a59183bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:13.974863 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042114fc-f91d-47e7-836a-8a4aab2bf33a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:13.975812 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042114fc-f91d-47e7-836a-8a4aab2bf33a service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:13.978135 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042114fc-f91d-47e7-836a-8a4aab2bf33a service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:13.978670 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042114fc-f91d-47e7-836a-8a4aab2bf33a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:14.001092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-80f220c9-0ba2-4604-8f8c-fdd9a59183bd None None] MFA Rules not processed for user `3ac7391a3c41499d94e5f97478797313`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:14.005783 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-80f220c9-0ba2-4604-8f8c-fdd9a59183bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:14.007497 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042114fc-f91d-47e7-836a-8a4aab2bf33a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:14.009661 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042114fc-f91d-47e7-836a-8a4aab2bf33a service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:14.025766 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 819/1635] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jun 29 12:33:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:14.037307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 817/1636] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 12:33:14.119254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f220c9-0ba2-4604-8f8c-fdd9a59183bd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:14.150690 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f220c9-0ba2-4604-8f8c-fdd9a59183bd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:14.165900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3aec9c-af00-4c0e-bc81-420c103ac1a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:14.167378 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3aec9c-af00-4c0e-bc81-420c103ac1a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:14.167679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3aec9c-af00-4c0e-bc81-420c103ac1a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:14.168059 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3aec9c-af00-4c0e-bc81-420c103ac1a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:14.191710 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3aec9c-af00-4c0e-bc81-420c103ac1a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:14.194381 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3aec9c-af00-4c0e-bc81-420c103ac1a2 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:14.217397 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 820/1637] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:14.604109 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042114fc-f91d-47e7-836a-8a4aab2bf33a service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:14.755595 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042114fc-f91d-47e7-836a-8a4aab2bf33a service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:14.771427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f52d0-e6e0-4a51-bc38-47c00ba13490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:14.772979 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f52d0-e6e0-4a51-bc38-47c00ba13490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:14.773877 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f52d0-e6e0-4a51-bc38-47c00ba13490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:14.774566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f52d0-e6e0-4a51-bc38-47c00ba13490 admin admin] PATH_INFO: `/v3/users/08d02203c5654d71853045704fb0d914` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:14.778555 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f52d0-e6e0-4a51-bc38-47c00ba13490 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08d02203c5654d71853045704fb0d914)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:14.782375 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-229f52d0-e6e0-4a51-bc38-47c00ba13490 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 12:33:14.782375 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jun 29 12:33:14.782375 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: Jun 29 12:33:14.787114 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f52d0-e6e0-4a51-bc38-47c00ba13490 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:15.016991 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-229f52d0-e6e0-4a51-bc38-47c00ba13490 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08d02203c5654d71853045704fb0d914'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:15.039403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-229f52d0-e6e0-4a51-bc38-47c00ba13490 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08d02203c5654d71853045704fb0d914'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:15.118301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-229f52d0-e6e0-4a51-bc38-47c00ba13490 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08d02203c5654d71853045704fb0d914'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:15.123658 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-229f52d0-e6e0-4a51-bc38-47c00ba13490 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08d02203c5654d71853045704fb0d914'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:15.124446 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-229f52d0-e6e0-4a51-bc38-47c00ba13490 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08d02203c5654d71853045704fb0d914'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:15.145543 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 818/1638] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:33:14 2022] DELETE /identity/v3/users/08d02203c5654d71853045704fb0d914 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:15.150190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e6283-bf5d-4bc7-a2ed-8c5bf8baef6d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:15.150816 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e6283-bf5d-4bc7-a2ed-8c5bf8baef6d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:15.150816 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e6283-bf5d-4bc7-a2ed-8c5bf8baef6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:15.209035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6a1e6283-bf5d-4bc7-a2ed-8c5bf8baef6d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:15.210183 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6a1e6283-bf5d-4bc7-a2ed-8c5bf8baef6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:15.347369 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88387a-eca0-4498-8afa-192de1c58938 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:15.347751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88387a-eca0-4498-8afa-192de1c58938 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:15.348043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88387a-eca0-4498-8afa-192de1c58938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:15.349338 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 821/1639] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:15.390126 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e6283-bf5d-4bc7-a2ed-8c5bf8baef6d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:15.391548 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6a1e6283-bf5d-4bc7-a2ed-8c5bf8baef6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:15.429400 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ac88387a-eca0-4498-8afa-192de1c58938 None None] MFA Rules not processed for user `558836b50d72443d97d97597b5574954`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:15.432121 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ac88387a-eca0-4498-8afa-192de1c58938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:15.523674 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:15.532446 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e6283-bf5d-4bc7-a2ed-8c5bf8baef6d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:15.540144 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31105d-3aaa-4168-92cd-77bdfdfd375b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:15.541047 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31105d-3aaa-4168-92cd-77bdfdfd375b service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:15.542403 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31105d-3aaa-4168-92cd-77bdfdfd375b service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:15.542925 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd31105d-3aaa-4168-92cd-77bdfdfd375b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:15.570060 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd31105d-3aaa-4168-92cd-77bdfdfd375b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:15.571930 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd31105d-3aaa-4168-92cd-77bdfdfd375b service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:15.572154 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:15.598124 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 819/1640] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:15.600100 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88387a-eca0-4498-8afa-192de1c58938 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:15.602222 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ac88387a-eca0-4498-8afa-192de1c58938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:15.602893 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 822/1641] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 12:33:15.642920 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88387a-eca0-4498-8afa-192de1c58938 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:15.654386 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf300a-d8a8-4878-b070-4b7c4596dba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:15.655416 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf300a-d8a8-4878-b070-4b7c4596dba2 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:15.655885 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf300a-d8a8-4878-b070-4b7c4596dba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:15.656577 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbf300a-d8a8-4878-b070-4b7c4596dba2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:15.657655 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbf300a-d8a8-4878-b070-4b7c4596dba2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:15.659886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbf300a-d8a8-4878-b070-4b7c4596dba2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:15.721275 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 820/1642] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:15 2022] POST /identity/v3/projects => generated 405 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:15.726242 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31105d-3aaa-4168-92cd-77bdfdfd375b service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:15.743192 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd31105d-3aaa-4168-92cd-77bdfdfd375b service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:15.870345 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9088860-782e-4b35-b33e-b123f0251e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:15.871352 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9088860-782e-4b35-b33e-b123f0251e62 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:15.871916 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9088860-782e-4b35-b33e-b123f0251e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:15.872500 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9088860-782e-4b35-b33e-b123f0251e62 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:15.873133 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9088860-782e-4b35-b33e-b123f0251e62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:15.875626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9088860-782e-4b35-b33e-b123f0251e62 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:15.901356 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 823/1643] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:15 2022] POST /identity/v3/users => generated 309 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:15.905294 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf300a-d8a8-4878-b070-4b7c4596dba2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:15.920860 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbf300a-d8a8-4878-b070-4b7c4596dba2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:15.935627 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab1b1c-827d-47c1-9753-af79ba1e362b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:15.936334 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab1b1c-827d-47c1-9753-af79ba1e362b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:15.936625 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab1b1c-827d-47c1-9753-af79ba1e362b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:15.936908 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab1b1c-827d-47c1-9753-af79ba1e362b admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:15.937667 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab1b1c-827d-47c1-9753-af79ba1e362b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:15.939362 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab1b1c-827d-47c1-9753-af79ba1e362b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:15.971487 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 821/1644] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:15 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:15.973032 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9088860-782e-4b35-b33e-b123f0251e62 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:15.985819 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9088860-782e-4b35-b33e-b123f0251e62 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:16.000995 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e09768-3d7d-4ec2-a72e-ec12bfc5835c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:16.002322 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e09768-3d7d-4ec2-a72e-ec12bfc5835c admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:16.005303 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e09768-3d7d-4ec2-a72e-ec12bfc5835c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:16.005948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e09768-3d7d-4ec2-a72e-ec12bfc5835c admin admin] PATH_INFO: `/v3/projects/f08b62f392d0466892d2662cf4949261/users/9a58575aafa8411f9c6f843d4c176896/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:16.051006 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e09768-3d7d-4ec2-a72e-ec12bfc5835c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f08b62f392d0466892d2662cf4949261, user_id=9a58575aafa8411f9c6f843d4c176896, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:16.055783 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e09768-3d7d-4ec2-a72e-ec12bfc5835c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:16.090710 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 824/1645] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:15 2022] PUT /identity/v3/projects/f08b62f392d0466892d2662cf4949261/users/9a58575aafa8411f9c6f843d4c176896/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:16.092011 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab1b1c-827d-47c1-9753-af79ba1e362b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:16.110714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab1b1c-827d-47c1-9753-af79ba1e362b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:16.274368 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57913820-ce0c-48c1-92ca-b08ec40f994c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:16.275357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57913820-ce0c-48c1-92ca-b08ec40f994c admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:16.276390 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57913820-ce0c-48c1-92ca-b08ec40f994c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:16.276853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57913820-ce0c-48c1-92ca-b08ec40f994c admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:16.277758 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57913820-ce0c-48c1-92ca-b08ec40f994c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:16.279594 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57913820-ce0c-48c1-92ca-b08ec40f994c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:16.306334 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 822/1646] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:16 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:16.313000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e09768-3d7d-4ec2-a72e-ec12bfc5835c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:16.337465 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e09768-3d7d-4ec2-a72e-ec12bfc5835c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:16.347836 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5ebb3-6fe8-4d0c-8b76-dda4097edcd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:16.349560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5ebb3-6fe8-4d0c-8b76-dda4097edcd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:16.350004 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5ebb3-6fe8-4d0c-8b76-dda4097edcd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:16.350464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5ebb3-6fe8-4d0c-8b76-dda4097edcd7 admin admin] PATH_INFO: `/v3/projects/f08b62f392d0466892d2662cf4949261/users/9a58575aafa8411f9c6f843d4c176896/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:16.354168 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe5ebb3-6fe8-4d0c-8b76-dda4097edcd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f08b62f392d0466892d2662cf4949261, user_id=9a58575aafa8411f9c6f843d4c176896, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:16.357386 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe5ebb3-6fe8-4d0c-8b76-dda4097edcd7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:16.382949 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 825/1647] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:16 2022] PUT /identity/v3/projects/f08b62f392d0466892d2662cf4949261/users/9a58575aafa8411f9c6f843d4c176896/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:16.829562 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57913820-ce0c-48c1-92ca-b08ec40f994c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:16.831659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-57913820-ce0c-48c1-92ca-b08ec40f994c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:16.856480 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57913820-ce0c-48c1-92ca-b08ec40f994c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:17.018619 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4bbc3d-c95a-40f8-9af2-f8eb48fdca44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:17.019506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4bbc3d-c95a-40f8-9af2-f8eb48fdca44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:17.019880 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4bbc3d-c95a-40f8-9af2-f8eb48fdca44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:17.020060 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4bbc3d-c95a-40f8-9af2-f8eb48fdca44 admin admin] PATH_INFO: `/v3/projects/c8efd9db53c14bf69dc0ebd8ebac75fc` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:17.023325 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4bbc3d-c95a-40f8-9af2-f8eb48fdca44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8efd9db53c14bf69dc0ebd8ebac75fc)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:17.025323 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4bbc3d-c95a-40f8-9af2-f8eb48fdca44 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:17.119890 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bc4bbc3d-c95a-40f8-9af2-f8eb48fdca44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8efd9db53c14bf69dc0ebd8ebac75fc', 'request_id': 'req-bc4bbc3d-c95a-40f8-9af2-f8eb48fdca44'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:17.169971 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bc4bbc3d-c95a-40f8-9af2-f8eb48fdca44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8efd9db53c14bf69dc0ebd8ebac75fc', 'request_id': 'req-bc4bbc3d-c95a-40f8-9af2-f8eb48fdca44'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:17.169971 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:17.169971 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bc4bbc3d-c95a-40f8-9af2-f8eb48fdca44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8efd9db53c14bf69dc0ebd8ebac75fc', 'request_id': 'req-bc4bbc3d-c95a-40f8-9af2-f8eb48fdca44'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:17.185805 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5ebb3-6fe8-4d0c-8b76-dda4097edcd7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:17.188833 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bfe5ebb3-6fe8-4d0c-8b76-dda4097edcd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:17.198755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bc4bbc3d-c95a-40f8-9af2-f8eb48fdca44 admin admin] The token cache is being invalidate because project c8efd9db53c14bf69dc0ebd8ebac75fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:33:17.199060 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bc4bbc3d-c95a-40f8-9af2-f8eb48fdca44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:17.199843 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bc4bbc3d-c95a-40f8-9af2-f8eb48fdca44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:17.202826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 823/1648] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:33:16 2022] DELETE /identity/v3/projects/c8efd9db53c14bf69dc0ebd8ebac75fc => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:17.209905 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790bc249-3698-4d94-b6f1-e50739ab3f98 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:17.214259 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790bc249-3698-4d94-b6f1-e50739ab3f98 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:17.214259 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790bc249-3698-4d94-b6f1-e50739ab3f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:17.216728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5ebb3-6fe8-4d0c-8b76-dda4097edcd7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:17.258054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-790bc249-3698-4d94-b6f1-e50739ab3f98 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:17.260796 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-790bc249-3698-4d94-b6f1-e50739ab3f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:17.369579 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583da8e-44ab-4c2d-86c9-feda6c66afe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:17.371602 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583da8e-44ab-4c2d-86c9-feda6c66afe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:17.372244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583da8e-44ab-4c2d-86c9-feda6c66afe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:17.373281 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583da8e-44ab-4c2d-86c9-feda6c66afe5 admin admin] PATH_INFO: `/v3/users/aa9c3da38c7e44c2892b117c204ca7bc` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:17.376786 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3583da8e-44ab-4c2d-86c9-feda6c66afe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa9c3da38c7e44c2892b117c204ca7bc)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:17.383327 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3583da8e-44ab-4c2d-86c9-feda6c66afe5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:17.418585 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 824/1649] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:17.424814 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef90a1-6479-47b1-8a30-e7bacbd48ca3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:17.425111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef90a1-6479-47b1-8a30-e7bacbd48ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:17.425388 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ef90a1-6479-47b1-8a30-e7bacbd48ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:17.464811 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-07ef90a1-6479-47b1-8a30-e7bacbd48ca3 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:17.467323 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-07ef90a1-6479-47b1-8a30-e7bacbd48ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:17.486904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 825/1650] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:33:17.491333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef90a1-6479-47b1-8a30-e7bacbd48ca3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:17.495065 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:17.499538 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3583da8e-44ab-4c2d-86c9-feda6c66afe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa9c3da38c7e44c2892b117c204ca7bc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:17.500786 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3583da8e-44ab-4c2d-86c9-feda6c66afe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa9c3da38c7e44c2892b117c204ca7bc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:17.509310 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ef90a1-6479-47b1-8a30-e7bacbd48ca3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:17.523969 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8f6d1-65a1-45f9-8cd8-e7f131b5054b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:17.524213 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8f6d1-65a1-45f9-8cd8-e7f131b5054b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:17.524500 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8f6d1-65a1-45f9-8cd8-e7f131b5054b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:17.524762 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8f6d1-65a1-45f9-8cd8-e7f131b5054b admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:17.525498 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a8f6d1-65a1-45f9-8cd8-e7f131b5054b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:33:17.526158 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a8f6d1-65a1-45f9-8cd8-e7f131b5054b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:17.527574 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a8f6d1-65a1-45f9-8cd8-e7f131b5054b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:17.541451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3583da8e-44ab-4c2d-86c9-feda6c66afe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa9c3da38c7e44c2892b117c204ca7bc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:17.542233 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3583da8e-44ab-4c2d-86c9-feda6c66afe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa9c3da38c7e44c2892b117c204ca7bc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:17.555597 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 826/1651] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:33:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:33:17.560948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8f6d1-65a1-45f9-8cd8-e7f131b5054b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:17.561477 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3583da8e-44ab-4c2d-86c9-feda6c66afe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa9c3da38c7e44c2892b117c204ca7bc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:17.564259 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d3a8f6d1-65a1-45f9-8cd8-e7f131b5054b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:17.584467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8f6d1-65a1-45f9-8cd8-e7f131b5054b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:17.586001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 826/1652] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:33:17 2022] DELETE /identity/v3/users/aa9c3da38c7e44c2892b117c204ca7bc => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:17.590687 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dedd33-7c68-4665-833e-6f9dd865f3e4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:17.591040 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dedd33-7c68-4665-833e-6f9dd865f3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:17.591331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dedd33-7c68-4665-833e-6f9dd865f3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:17.621813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-40dedd33-7c68-4665-833e-6f9dd865f3e4 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:17.623116 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-40dedd33-7c68-4665-833e-6f9dd865f3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:17.743391 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 827/1653] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:33:17.743391 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dedd33-7c68-4665-833e-6f9dd865f3e4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:17.745453 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-40dedd33-7c68-4665-833e-6f9dd865f3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:17.776540 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dedd33-7c68-4665-833e-6f9dd865f3e4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:17.783380 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709bd32e-5ee0-4d4e-adae-779d41fc20f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:17.786949 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709bd32e-5ee0-4d4e-adae-779d41fc20f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:17.787258 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709bd32e-5ee0-4d4e-adae-779d41fc20f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:17.787551 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709bd32e-5ee0-4d4e-adae-779d41fc20f5 admin admin] PATH_INFO: `/v3/users/39886efe36ce4ac9b55f06dffd54d1a3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:17.788877 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709bd32e-5ee0-4d4e-adae-779d41fc20f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39886efe36ce4ac9b55f06dffd54d1a3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:17.791835 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709bd32e-5ee0-4d4e-adae-779d41fc20f5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:17.794602 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a3f515-ddf9-4076-86be-cd7022ad094e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:17.795458 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a3f515-ddf9-4076-86be-cd7022ad094e admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:17.796788 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a3f515-ddf9-4076-86be-cd7022ad094e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:17.798178 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a3f515-ddf9-4076-86be-cd7022ad094e admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:17.803391 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a3f515-ddf9-4076-86be-cd7022ad094e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:17.808058 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a3f515-ddf9-4076-86be-cd7022ad094e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:17.870948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 827/1654] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:17 2022] POST /identity/v3/projects => generated 381 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:17.876114 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a3f515-ddf9-4076-86be-cd7022ad094e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:17.882274 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-43a3f515-ddf9-4076-86be-cd7022ad094e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:17.907209 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-709bd32e-5ee0-4d4e-adae-779d41fc20f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39886efe36ce4ac9b55f06dffd54d1a3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:17.909229 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-709bd32e-5ee0-4d4e-adae-779d41fc20f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39886efe36ce4ac9b55f06dffd54d1a3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:17.947811 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-709bd32e-5ee0-4d4e-adae-779d41fc20f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39886efe36ce4ac9b55f06dffd54d1a3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:17.948643 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-709bd32e-5ee0-4d4e-adae-779d41fc20f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39886efe36ce4ac9b55f06dffd54d1a3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:17.970259 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-709bd32e-5ee0-4d4e-adae-779d41fc20f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39886efe36ce4ac9b55f06dffd54d1a3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:18.000117 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 828/1655] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:33:17 2022] DELETE /identity/v3/users/39886efe36ce4ac9b55f06dffd54d1a3 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:18.001121 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709bd32e-5ee0-4d4e-adae-779d41fc20f5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:18.002738 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-709bd32e-5ee0-4d4e-adae-779d41fc20f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:18.018374 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a3f515-ddf9-4076-86be-cd7022ad094e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:18.029336 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:18.102625 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:18.131920 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef9e059-24ee-46e6-976f-b67391df3eee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:18.133726 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9e059-24ee-46e6-976f-b67391df3eee service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:18.134340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9e059-24ee-46e6-976f-b67391df3eee service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:18.136148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9e059-24ee-46e6-976f-b67391df3eee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:18.155181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1ef9e059-24ee-46e6-976f-b67391df3eee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:18.176817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709bd32e-5ee0-4d4e-adae-779d41fc20f5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:18.202663 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f78b65-0df7-43f6-980e-1e4d914797fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:18.203359 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f78b65-0df7-43f6-980e-1e4d914797fc admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:18.204933 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef9e059-24ee-46e6-976f-b67391df3eee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:18.206606 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f78b65-0df7-43f6-980e-1e4d914797fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:18.206901 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f78b65-0df7-43f6-980e-1e4d914797fc admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:18.207631 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f78b65-0df7-43f6-980e-1e4d914797fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:18.209605 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef9e059-24ee-46e6-976f-b67391df3eee service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:18.214415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f78b65-0df7-43f6-980e-1e4d914797fc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:18.227281 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 828/1656] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:18.228715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd35563-922b-4da1-82f3-21b09285b5a8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:18.231844 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd35563-922b-4da1-82f3-21b09285b5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:18.232264 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd35563-922b-4da1-82f3-21b09285b5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:18.266917 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9fd35563-922b-4da1-82f3-21b09285b5a8 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:18.268227 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 829/1657] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:18 2022] POST /identity/v3/users => generated 291 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:18.271370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9fd35563-922b-4da1-82f3-21b09285b5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:18.271794 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f78b65-0df7-43f6-980e-1e4d914797fc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:18.289177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f78b65-0df7-43f6-980e-1e4d914797fc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:18.292496 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 829/1658] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:18.296090 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac51a23-fd6e-4585-a20b-b3d8e13ae42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:18.296950 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac51a23-fd6e-4585-a20b-b3d8e13ae42d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:18.297373 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac51a23-fd6e-4585-a20b-b3d8e13ae42d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:18.297783 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac51a23-fd6e-4585-a20b-b3d8e13ae42d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:18.299047 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac51a23-fd6e-4585-a20b-b3d8e13ae42d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:18.301095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac51a23-fd6e-4585-a20b-b3d8e13ae42d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:18.315417 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 830/1659] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:18.319274 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd35563-922b-4da1-82f3-21b09285b5a8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:18.340156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd35563-922b-4da1-82f3-21b09285b5a8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:18.351137 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2db5a0-4de6-4d57-aa10-1ee0e20f23fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:18.352746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2db5a0-4de6-4d57-aa10-1ee0e20f23fe admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:18.353915 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2db5a0-4de6-4d57-aa10-1ee0e20f23fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:18.354569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2db5a0-4de6-4d57-aa10-1ee0e20f23fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:18.356335 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2db5a0-4de6-4d57-aa10-1ee0e20f23fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:18.358527 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2db5a0-4de6-4d57-aa10-1ee0e20f23fe admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:18.377448 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac51a23-fd6e-4585-a20b-b3d8e13ae42d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:18.382412 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cac51a23-fd6e-4585-a20b-b3d8e13ae42d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:18.447322 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac51a23-fd6e-4585-a20b-b3d8e13ae42d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:18.450064 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 830/1660] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:18.453300 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2db5a0-4de6-4d57-aa10-1ee0e20f23fe admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:18.476644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614560d9-a4a3-4bc1-8d78-132518e7740a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:18.477338 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614560d9-a4a3-4bc1-8d78-132518e7740a service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:18.480524 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614560d9-a4a3-4bc1-8d78-132518e7740a service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:18.480970 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614560d9-a4a3-4bc1-8d78-132518e7740a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:18.509575 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2db5a0-4de6-4d57-aa10-1ee0e20f23fe admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:18.530301 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614560d9-a4a3-4bc1-8d78-132518e7740a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:18.531852 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614560d9-a4a3-4bc1-8d78-132518e7740a service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:18.537209 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a9a37-e4a1-45bb-8148-1b15e4927dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:18.542480 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a9a37-e4a1-45bb-8148-1b15e4927dec admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:18.542685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a9a37-e4a1-45bb-8148-1b15e4927dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:18.542917 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a9a37-e4a1-45bb-8148-1b15e4927dec admin admin] PATH_INFO: `/v3/projects/6d1e265893c14eafa75e67723215de7e/users/669758a928794a86a45afe6ad3d78595/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:18.588449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 831/1661] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:18.606640 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a9a37-e4a1-45bb-8148-1b15e4927dec admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d1e265893c14eafa75e67723215de7e, user_id=669758a928794a86a45afe6ad3d78595, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:18.610277 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28a9a37-e4a1-45bb-8148-1b15e4927dec admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:18.656265 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f00bf-a318-48bb-a2bd-b507440d8a32 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:18.656488 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f00bf-a318-48bb-a2bd-b507440d8a32 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:18.656787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f00bf-a318-48bb-a2bd-b507440d8a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:18.662830 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 831/1662] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:18 2022] PUT /identity/v3/projects/6d1e265893c14eafa75e67723215de7e/users/669758a928794a86a45afe6ad3d78595/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:18.726932 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-409f00bf-a318-48bb-a2bd-b507440d8a32 None None] MFA Rules not processed for user `669758a928794a86a45afe6ad3d78595`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:18.728811 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-409f00bf-a318-48bb-a2bd-b507440d8a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:18.882753 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 832/1663] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:33:18.885897 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a9a37-e4a1-45bb-8148-1b15e4927dec admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:18.998154 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a9a37-e4a1-45bb-8148-1b15e4927dec admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:19.003879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604bbbc-8af5-4c3c-b3f6-c35bc64f6244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:19.004914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604bbbc-8af5-4c3c-b3f6-c35bc64f6244 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:19.005561 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604bbbc-8af5-4c3c-b3f6-c35bc64f6244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:19.006010 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604bbbc-8af5-4c3c-b3f6-c35bc64f6244 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:19.006938 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604bbbc-8af5-4c3c-b3f6-c35bc64f6244 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:19.009752 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604bbbc-8af5-4c3c-b3f6-c35bc64f6244 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:19.059180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 832/1664] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:18 2022] POST /identity/v3/projects => generated 381 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:19.063977 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f00bf-a318-48bb-a2bd-b507440d8a32 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:19.078946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f00bf-a318-48bb-a2bd-b507440d8a32 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:19.264269 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff848f97-4293-4e04-86a7-8566007db028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:19.266211 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff848f97-4293-4e04-86a7-8566007db028 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:19.266759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff848f97-4293-4e04-86a7-8566007db028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:19.267411 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff848f97-4293-4e04-86a7-8566007db028 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:19.268334 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff848f97-4293-4e04-86a7-8566007db028 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:19.271733 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff848f97-4293-4e04-86a7-8566007db028 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:19.314809 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 833/1665] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:19 2022] POST /identity/v3/users => generated 297 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:19.319125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604bbbc-8af5-4c3c-b3f6-c35bc64f6244 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:19.336932 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604bbbc-8af5-4c3c-b3f6-c35bc64f6244 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:19.363569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1929033b-0530-42de-912d-edafe9fd9c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:19.363569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929033b-0530-42de-912d-edafe9fd9c74 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:19.364284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929033b-0530-42de-912d-edafe9fd9c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:19.364470 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929033b-0530-42de-912d-edafe9fd9c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:19.365250 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1929033b-0530-42de-912d-edafe9fd9c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:19.366940 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1929033b-0530-42de-912d-edafe9fd9c74 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:19.390054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 833/1666] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:19.394480 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff848f97-4293-4e04-86a7-8566007db028 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:19.455019 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff848f97-4293-4e04-86a7-8566007db028 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:19.481030 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb3432-e696-45d8-b4b2-464d42ea755d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:19.481944 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb3432-e696-45d8-b4b2-464d42ea755d admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:19.482717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb3432-e696-45d8-b4b2-464d42ea755d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:19.482717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb3432-e696-45d8-b4b2-464d42ea755d admin admin] PATH_INFO: `/v3/projects/e55414dc89c54939829e0ab66e8f37a7/users/b3046ca9fef445489e588f239e9e4ad1/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:19.535876 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb3432-e696-45d8-b4b2-464d42ea755d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e55414dc89c54939829e0ab66e8f37a7, user_id=b3046ca9fef445489e588f239e9e4ad1, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:19.541481 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb3432-e696-45d8-b4b2-464d42ea755d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:19.576148 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:19.596867 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 834/1667] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:19 2022] PUT /identity/v3/projects/e55414dc89c54939829e0ab66e8f37a7/users/b3046ca9fef445489e588f239e9e4ad1/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:19.602276 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1929033b-0530-42de-912d-edafe9fd9c74 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:19.617838 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1929033b-0530-42de-912d-edafe9fd9c74 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:19.755633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:19.800034 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908d34d-7086-4730-ae23-5bd1da842604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:19.800733 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908d34d-7086-4730-ae23-5bd1da842604 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:19.801018 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908d34d-7086-4730-ae23-5bd1da842604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:19.801295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908d34d-7086-4730-ae23-5bd1da842604 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:19.802261 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a908d34d-7086-4730-ae23-5bd1da842604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:19.804195 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a908d34d-7086-4730-ae23-5bd1da842604 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:19.826574 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 834/1668] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:33:19.831026 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb3432-e696-45d8-b4b2-464d42ea755d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:19.845533 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb3432-e696-45d8-b4b2-464d42ea755d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:19.852096 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f153e3-f84f-4a1d-8484-e1d55e117d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:19.853016 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f153e3-f84f-4a1d-8484-e1d55e117d89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:19.853311 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f153e3-f84f-4a1d-8484-e1d55e117d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:19.853580 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f153e3-f84f-4a1d-8484-e1d55e117d89 admin admin] PATH_INFO: `/v3/projects/e55414dc89c54939829e0ab66e8f37a7/users/b3046ca9fef445489e588f239e9e4ad1/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:19.855693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f153e3-f84f-4a1d-8484-e1d55e117d89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e55414dc89c54939829e0ab66e8f37a7, user_id=b3046ca9fef445489e588f239e9e4ad1, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:19.858790 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f153e3-f84f-4a1d-8484-e1d55e117d89 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:19.888843 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 835/1669] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:19 2022] PUT /identity/v3/projects/e55414dc89c54939829e0ab66e8f37a7/users/b3046ca9fef445489e588f239e9e4ad1/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:19.890526 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfd83a-6817-467a-b7f9-1af6ca0929d6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:19.890977 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfd83a-6817-467a-b7f9-1af6ca0929d6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:19.891488 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bfd83a-6817-467a-b7f9-1af6ca0929d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:19.942438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-74bfd83a-6817-467a-b7f9-1af6ca0929d6 None None] MFA Rules not processed for user `b3046ca9fef445489e588f239e9e4ad1`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:19.943604 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-74bfd83a-6817-467a-b7f9-1af6ca0929d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:20.112653 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 835/1670] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:33:20.159306 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f153e3-f84f-4a1d-8484-e1d55e117d89 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:20.161443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-04f153e3-f84f-4a1d-8484-e1d55e117d89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:20.283102 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f153e3-f84f-4a1d-8484-e1d55e117d89 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:20.292441 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d915dabb-7e1a-4cce-a943-2d69cdb3e2eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:20.293175 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915dabb-7e1a-4cce-a943-2d69cdb3e2eb service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:20.293488 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915dabb-7e1a-4cce-a943-2d69cdb3e2eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:20.293767 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915dabb-7e1a-4cce-a943-2d69cdb3e2eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:20.320182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d915dabb-7e1a-4cce-a943-2d69cdb3e2eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:20.321730 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d915dabb-7e1a-4cce-a943-2d69cdb3e2eb service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:20.342462 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 836/1671] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:20 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 12:33:20.382042 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bfd83a-6817-467a-b7f9-1af6ca0929d6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:20.387784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-74bfd83a-6817-467a-b7f9-1af6ca0929d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:20.501502 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d915dabb-7e1a-4cce-a943-2d69cdb3e2eb service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:20.519211 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d915dabb-7e1a-4cce-a943-2d69cdb3e2eb service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:20.552798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505113d2-f3c4-4abb-b6e4-b6b8e7cabf50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:20.553700 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505113d2-f3c4-4abb-b6e4-b6b8e7cabf50 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:20.555168 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505113d2-f3c4-4abb-b6e4-b6b8e7cabf50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:20.557375 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505113d2-f3c4-4abb-b6e4-b6b8e7cabf50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:20.571111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bfd83a-6817-467a-b7f9-1af6ca0929d6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:20.583458 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7746237-3068-4459-9e1f-959dd577ffa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:20.584329 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7746237-3068-4459-9e1f-959dd577ffa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:20.584615 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7746237-3068-4459-9e1f-959dd577ffa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:20.584896 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7746237-3068-4459-9e1f-959dd577ffa1 admin admin] PATH_INFO: `/v3/projects/1b65b9e1603349d2880eb2b4d793211e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:20.590401 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7746237-3068-4459-9e1f-959dd577ffa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b65b9e1603349d2880eb2b4d793211e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:20.592537 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7746237-3068-4459-9e1f-959dd577ffa1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:20.727474 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505113d2-f3c4-4abb-b6e4-b6b8e7cabf50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:20.728964 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505113d2-f3c4-4abb-b6e4-b6b8e7cabf50 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:20.730321 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f7746237-3068-4459-9e1f-959dd577ffa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b65b9e1603349d2880eb2b4d793211e', 'request_id': 'req-f7746237-3068-4459-9e1f-959dd577ffa1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:20.745380 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 837/1672] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:20.750993 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f7746237-3068-4459-9e1f-959dd577ffa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b65b9e1603349d2880eb2b4d793211e', 'request_id': 'req-f7746237-3068-4459-9e1f-959dd577ffa1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:20.761658 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f7746237-3068-4459-9e1f-959dd577ffa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b65b9e1603349d2880eb2b4d793211e', 'request_id': 'req-f7746237-3068-4459-9e1f-959dd577ffa1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:20.790284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:20.801744 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f7746237-3068-4459-9e1f-959dd577ffa1 admin admin] The token cache is being invalidate because project 1b65b9e1603349d2880eb2b4d793211e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:33:20.804098 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f7746237-3068-4459-9e1f-959dd577ffa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:20.805815 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f7746237-3068-4459-9e1f-959dd577ffa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:20.813553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 836/1673] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:33:20 2022] DELETE /identity/v3/projects/1b65b9e1603349d2880eb2b4d793211e => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:20.818973 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505113d2-f3c4-4abb-b6e4-b6b8e7cabf50 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:20.842644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505113d2-f3c4-4abb-b6e4-b6b8e7cabf50 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:21.089263 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85251f68-1318-4e83-93a6-9ad71e511205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:21.090193 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85251f68-1318-4e83-93a6-9ad71e511205 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:21.090513 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85251f68-1318-4e83-93a6-9ad71e511205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:21.090797 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85251f68-1318-4e83-93a6-9ad71e511205 admin admin] PATH_INFO: `/v3/users/bb55a30c00f24cb995c08c9d057cc567` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:21.093430 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85251f68-1318-4e83-93a6-9ad71e511205 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb55a30c00f24cb995c08c9d057cc567)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:21.097421 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85251f68-1318-4e83-93a6-9ad71e511205 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:21.237065 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:21.295201 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-85251f68-1318-4e83-93a6-9ad71e511205 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb55a30c00f24cb995c08c9d057cc567'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:21.296943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-85251f68-1318-4e83-93a6-9ad71e511205 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb55a30c00f24cb995c08c9d057cc567'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:21.327525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-85251f68-1318-4e83-93a6-9ad71e511205 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb55a30c00f24cb995c08c9d057cc567'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:21.331070 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-85251f68-1318-4e83-93a6-9ad71e511205 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb55a30c00f24cb995c08c9d057cc567'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:21.363370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-85251f68-1318-4e83-93a6-9ad71e511205 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb55a30c00f24cb995c08c9d057cc567'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:21.397044 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7746237-3068-4459-9e1f-959dd577ffa1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:21.398606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f7746237-3068-4459-9e1f-959dd577ffa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:21.401935 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 838/1674] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:33:20 2022] DELETE /identity/v3/users/bb55a30c00f24cb995c08c9d057cc567 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:21.422476 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7746237-3068-4459-9e1f-959dd577ffa1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:21.618308 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ef89ec-21f3-497f-b3db-4a49adaa32d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:21.619463 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef89ec-21f3-497f-b3db-4a49adaa32d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:21.619888 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef89ec-21f3-497f-b3db-4a49adaa32d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:21.620305 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ef89ec-21f3-497f-b3db-4a49adaa32d1 admin admin] PATH_INFO: `/v3/projects/6fd787ac66684a5b953f94e1273a75e4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:21.626621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef89ec-21f3-497f-b3db-4a49adaa32d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fd787ac66684a5b953f94e1273a75e4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:21.628606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ef89ec-21f3-497f-b3db-4a49adaa32d1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:21.744968 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-95ef89ec-21f3-497f-b3db-4a49adaa32d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fd787ac66684a5b953f94e1273a75e4', 'request_id': 'req-95ef89ec-21f3-497f-b3db-4a49adaa32d1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:21.757780 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-95ef89ec-21f3-497f-b3db-4a49adaa32d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fd787ac66684a5b953f94e1273a75e4', 'request_id': 'req-95ef89ec-21f3-497f-b3db-4a49adaa32d1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:21.777398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-95ef89ec-21f3-497f-b3db-4a49adaa32d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fd787ac66684a5b953f94e1273a75e4', 'request_id': 'req-95ef89ec-21f3-497f-b3db-4a49adaa32d1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:21.800293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-95ef89ec-21f3-497f-b3db-4a49adaa32d1 admin admin] The token cache is being invalidate because project 6fd787ac66684a5b953f94e1273a75e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:33:21.800948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-95ef89ec-21f3-497f-b3db-4a49adaa32d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:21.802905 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-95ef89ec-21f3-497f-b3db-4a49adaa32d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:21.804904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 837/1675] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:33:21 2022] DELETE /identity/v3/projects/6fd787ac66684a5b953f94e1273a75e4 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:33:21.810169 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1883ec-6e53-4553-b8d4-214e825f6279 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:21.811127 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1883ec-6e53-4553-b8d4-214e825f6279 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:21.811579 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1883ec-6e53-4553-b8d4-214e825f6279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:21.836133 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fc1883ec-6e53-4553-b8d4-214e825f6279 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:21.837647 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fc1883ec-6e53-4553-b8d4-214e825f6279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:21.990409 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 839/1676] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:21.998661 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f573277-e883-4158-b40f-35a26d0e16aa None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:21.999042 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f573277-e883-4158-b40f-35a26d0e16aa None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:21.999414 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f573277-e883-4158-b40f-35a26d0e16aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:22.030899 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4f573277-e883-4158-b40f-35a26d0e16aa None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:22.032150 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4f573277-e883-4158-b40f-35a26d0e16aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:22.043698 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 838/1677] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:22.047719 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1883ec-6e53-4553-b8d4-214e825f6279 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:22.060329 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1883ec-6e53-4553-b8d4-214e825f6279 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:22.070290 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f684f9-7704-4f06-81f5-ec28caae2f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:22.072144 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f684f9-7704-4f06-81f5-ec28caae2f0a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:22.072523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f684f9-7704-4f06-81f5-ec28caae2f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:22.072882 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f684f9-7704-4f06-81f5-ec28caae2f0a admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:22.073730 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f684f9-7704-4f06-81f5-ec28caae2f0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:33:22.074155 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f684f9-7704-4f06-81f5-ec28caae2f0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:22.076043 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f684f9-7704-4f06-81f5-ec28caae2f0a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:22.095718 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 840/1678] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:33:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:33:22.099980 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f573277-e883-4158-b40f-35a26d0e16aa None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:22.112434 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f573277-e883-4158-b40f-35a26d0e16aa None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:22.122090 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5c991-f56e-4505-a165-8db29f08706a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:22.123498 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5c991-f56e-4505-a165-8db29f08706a admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:22.123946 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5c991-f56e-4505-a165-8db29f08706a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:22.124331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5c991-f56e-4505-a165-8db29f08706a admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:22.125165 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c5c991-f56e-4505-a165-8db29f08706a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:22.129111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c5c991-f56e-4505-a165-8db29f08706a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:22.193486 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 839/1679] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:22 2022] POST /identity/v3/projects => generated 405 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:22.197177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f684f9-7704-4f06-81f5-ec28caae2f0a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:22.213374 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f684f9-7704-4f06-81f5-ec28caae2f0a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:22.390558 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d8199-1018-40dc-91f9-f8e6495c3407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:22.392000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d8199-1018-40dc-91f9-f8e6495c3407 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:22.392391 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d8199-1018-40dc-91f9-f8e6495c3407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:22.394549 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d8199-1018-40dc-91f9-f8e6495c3407 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:22.396318 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425d8199-1018-40dc-91f9-f8e6495c3407 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:22.400080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425d8199-1018-40dc-91f9-f8e6495c3407 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:22.440088 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 841/1680] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:22 2022] POST /identity/v3/users => generated 303 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:33:22.449479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5c991-f56e-4505-a165-8db29f08706a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:22.456111 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a00d39-1fbc-4288-9e58-7c3593cc806d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:22.456751 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a00d39-1fbc-4288-9e58-7c3593cc806d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:22.459649 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a00d39-1fbc-4288-9e58-7c3593cc806d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:22.465501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5c991-f56e-4505-a165-8db29f08706a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:22.481814 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f767f69-05a6-4457-9a62-244d468b9923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:22.485896 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f767f69-05a6-4457-9a62-244d468b9923 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:22.486253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f767f69-05a6-4457-9a62-244d468b9923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:22.489935 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f767f69-05a6-4457-9a62-244d468b9923 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:22.492665 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f767f69-05a6-4457-9a62-244d468b9923 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:22.496772 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f767f69-05a6-4457-9a62-244d468b9923 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:22.527950 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 840/1681] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:22.531021 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394e8fd-0d36-4670-aca6-f5cb923033da None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:22.531505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394e8fd-0d36-4670-aca6-f5cb923033da None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:22.534748 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394e8fd-0d36-4670-aca6-f5cb923033da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:22.564698 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-50a00d39-1fbc-4288-9e58-7c3593cc806d None None] MFA Rules not processed for user `6252d0519ed9421e89710ca5294524c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:22.566491 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-50a00d39-1fbc-4288-9e58-7c3593cc806d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:22.586319 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:22.608604 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2394e8fd-0d36-4670-aca6-f5cb923033da None None] MFA Rules not processed for user `9a58575aafa8411f9c6f843d4c176896`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:22.610211 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2394e8fd-0d36-4670-aca6-f5cb923033da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:22.731671 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:22.743423 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 842/1682] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:22.744181 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a00d39-1fbc-4288-9e58-7c3593cc806d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:22.775854 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a00d39-1fbc-4288-9e58-7c3593cc806d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:22.801571 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f649c4-71f0-4e39-998b-11ac8323a56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:22.802123 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f649c4-71f0-4e39-998b-11ac8323a56c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:22.802406 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f649c4-71f0-4e39-998b-11ac8323a56c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:22.802707 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f649c4-71f0-4e39-998b-11ac8323a56c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:22.803621 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f649c4-71f0-4e39-998b-11ac8323a56c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:22.805127 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f649c4-71f0-4e39-998b-11ac8323a56c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:22.828492 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 843/1683] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:22.836446 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f649c4-71f0-4e39-998b-11ac8323a56c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:22.836616 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 841/1684] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:22 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:22.837826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394e8fd-0d36-4670-aca6-f5cb923033da None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:22.842311 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a3f649c4-71f0-4e39-998b-11ac8323a56c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:22.874380 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394e8fd-0d36-4670-aca6-f5cb923033da None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:22.891341 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d74ac-3440-43df-abb1-c71d792335e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:22.892410 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d74ac-3440-43df-abb1-c71d792335e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:22.892866 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d74ac-3440-43df-abb1-c71d792335e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:22.893293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d74ac-3440-43df-abb1-c71d792335e9 admin admin] PATH_INFO: `/v3/projects/c9758340629f4e159b3a2fc279a76b98/users/8f8fcb31e8a14dadb3a9a77d670fe6cc/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:22.935309 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d74ac-3440-43df-abb1-c71d792335e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9758340629f4e159b3a2fc279a76b98, user_id=8f8fcb31e8a14dadb3a9a77d670fe6cc, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:22.942991 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d74ac-3440-43df-abb1-c71d792335e9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:22.987230 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 842/1685] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:22 2022] PUT /identity/v3/projects/c9758340629f4e159b3a2fc279a76b98/users/8f8fcb31e8a14dadb3a9a77d670fe6cc/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:23.002333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61176a01-bc31-4188-b9f4-02bbdda5fe5d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:23.002815 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61176a01-bc31-4188-b9f4-02bbdda5fe5d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:23.003240 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61176a01-bc31-4188-b9f4-02bbdda5fe5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:23.115833 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-61176a01-bc31-4188-b9f4-02bbdda5fe5d None None] MFA Rules not processed for user `8f8fcb31e8a14dadb3a9a77d670fe6cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:23.117352 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-61176a01-bc31-4188-b9f4-02bbdda5fe5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:23.123639 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f649c4-71f0-4e39-998b-11ac8323a56c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:23.288131 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811122bd-523f-4ad6-9b80-cb6986dd676d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:23.289061 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811122bd-523f-4ad6-9b80-cb6986dd676d service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:23.289496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811122bd-523f-4ad6-9b80-cb6986dd676d service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:23.289972 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811122bd-523f-4ad6-9b80-cb6986dd676d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:23.291486 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 843/1686] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:23 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:33:23.292420 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61176a01-bc31-4188-b9f4-02bbdda5fe5d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:23.346170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:23.415264 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61176a01-bc31-4188-b9f4-02bbdda5fe5d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:23.427965 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811122bd-523f-4ad6-9b80-cb6986dd676d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:23.429936 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811122bd-523f-4ad6-9b80-cb6986dd676d service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:23.430134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d16214-7c21-4834-a2c1-6ae89bc3186c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:23.430974 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d16214-7c21-4834-a2c1-6ae89bc3186c admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:23.431823 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d16214-7c21-4834-a2c1-6ae89bc3186c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:23.432125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d16214-7c21-4834-a2c1-6ae89bc3186c admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:23.432844 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d16214-7c21-4834-a2c1-6ae89bc3186c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:23.439940 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d16214-7c21-4834-a2c1-6ae89bc3186c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:23.453704 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 844/1687] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:23.501405 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 844/1688] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:23 2022] POST /identity/v3/projects => generated 403 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:23.505496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811122bd-523f-4ad6-9b80-cb6986dd676d service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:23.521745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811122bd-523f-4ad6-9b80-cb6986dd676d service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:23.658325 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f8d06-07e5-4cf7-a272-c411309de636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:23.660568 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f8d06-07e5-4cf7-a272-c411309de636 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:23.661383 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f8d06-07e5-4cf7-a272-c411309de636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:23.661722 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f8d06-07e5-4cf7-a272-c411309de636 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:23.667331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f8d06-07e5-4cf7-a272-c411309de636 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:23.669327 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f8d06-07e5-4cf7-a272-c411309de636 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:23.720183 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 845/1689] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:23 2022] POST /identity/v3/users => generated 308 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:23.721987 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d16214-7c21-4834-a2c1-6ae89bc3186c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:23.735362 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d16214-7c21-4834-a2c1-6ae89bc3186c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:23.746130 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248cc9a4-aae6-4443-b163-c657242fbe40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:23.746861 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248cc9a4-aae6-4443-b163-c657242fbe40 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:23.747131 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248cc9a4-aae6-4443-b163-c657242fbe40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:23.747416 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248cc9a4-aae6-4443-b163-c657242fbe40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:23.748245 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248cc9a4-aae6-4443-b163-c657242fbe40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:23.756264 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248cc9a4-aae6-4443-b163-c657242fbe40 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:23.786600 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 845/1690] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:23.791764 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f8d06-07e5-4cf7-a272-c411309de636 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:23.811115 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f8d06-07e5-4cf7-a272-c411309de636 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:23.818016 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d4d65b-636d-443f-b8af-603f0ea009a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:23.823101 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4d65b-636d-443f-b8af-603f0ea009a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:23.823402 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4d65b-636d-443f-b8af-603f0ea009a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:23.823695 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d4d65b-636d-443f-b8af-603f0ea009a0 admin admin] PATH_INFO: `/v3/projects/dda90d00caeb4656a8da048fb2039657/users/0ecc669e82b349168b87be347b50ba6a/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:23.864443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d4d65b-636d-443f-b8af-603f0ea009a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dda90d00caeb4656a8da048fb2039657, user_id=0ecc669e82b349168b87be347b50ba6a, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:23.867783 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d4d65b-636d-443f-b8af-603f0ea009a0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:23.901267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 846/1691] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:23 2022] PUT /identity/v3/projects/dda90d00caeb4656a8da048fb2039657/users/0ecc669e82b349168b87be347b50ba6a/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:23.905574 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248cc9a4-aae6-4443-b163-c657242fbe40 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:23.932937 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248cc9a4-aae6-4443-b163-c657242fbe40 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:23.943179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:24.067347 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338b7fd-f974-4c65-a995-6b7a6c7f759e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:24.068699 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338b7fd-f974-4c65-a995-6b7a6c7f759e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:24.069763 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338b7fd-f974-4c65-a995-6b7a6c7f759e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:24.070052 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0338b7fd-f974-4c65-a995-6b7a6c7f759e admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:24.071587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0338b7fd-f974-4c65-a995-6b7a6c7f759e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:24.076405 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0338b7fd-f974-4c65-a995-6b7a6c7f759e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:24.119584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d4d65b-636d-443f-b8af-603f0ea009a0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:24.120780 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 846/1692] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:24.125911 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338b7fd-f974-4c65-a995-6b7a6c7f759e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:24.132780 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0338b7fd-f974-4c65-a995-6b7a6c7f759e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:24.138622 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d4d65b-636d-443f-b8af-603f0ea009a0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:24.151130 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b367a69-6e2f-4dc8-b5ed-8e77abe40f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:24.152442 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b367a69-6e2f-4dc8-b5ed-8e77abe40f20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:24.152938 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b367a69-6e2f-4dc8-b5ed-8e77abe40f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:24.153564 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b367a69-6e2f-4dc8-b5ed-8e77abe40f20 admin admin] PATH_INFO: `/v3/projects/dda90d00caeb4656a8da048fb2039657/users/0ecc669e82b349168b87be347b50ba6a/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:24.159830 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b367a69-6e2f-4dc8-b5ed-8e77abe40f20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dda90d00caeb4656a8da048fb2039657, user_id=0ecc669e82b349168b87be347b50ba6a, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:24.166924 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b367a69-6e2f-4dc8-b5ed-8e77abe40f20 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:24.188897 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0338b7fd-f974-4c65-a995-6b7a6c7f759e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:24.207324 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8bd076-68be-4446-b587-575e4d57c843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:24.210324 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bd076-68be-4446-b587-575e4d57c843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:24.210671 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bd076-68be-4446-b587-575e4d57c843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:24.210960 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8bd076-68be-4446-b587-575e4d57c843 admin admin] PATH_INFO: `/v3/projects/587b93767a5645c99b26b4acd3600515` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:24.215647 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8bd076-68be-4446-b587-575e4d57c843 admin admin] RBAC: Authorizing `identity:delete_project(project_id=587b93767a5645c99b26b4acd3600515)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:24.218629 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 847/1693] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:24 2022] PUT /identity/v3/projects/dda90d00caeb4656a8da048fb2039657/users/0ecc669e82b349168b87be347b50ba6a/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:33:24.230453 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116611a5-038d-4ccf-bffc-178b9892deee None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:24.230759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116611a5-038d-4ccf-bffc-178b9892deee None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:24.231045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116611a5-038d-4ccf-bffc-178b9892deee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:24.233090 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8bd076-68be-4446-b587-575e4d57c843 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:24.322243 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-116611a5-038d-4ccf-bffc-178b9892deee None None] MFA Rules not processed for user `0ecc669e82b349168b87be347b50ba6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:24.330775 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-116611a5-038d-4ccf-bffc-178b9892deee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:24.443261 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ec8bd076-68be-4446-b587-575e4d57c843 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '587b93767a5645c99b26b4acd3600515', 'request_id': 'req-ec8bd076-68be-4446-b587-575e4d57c843'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:24.462860 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ec8bd076-68be-4446-b587-575e4d57c843 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '587b93767a5645c99b26b4acd3600515', 'request_id': 'req-ec8bd076-68be-4446-b587-575e4d57c843'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:24.487632 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ec8bd076-68be-4446-b587-575e4d57c843 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '587b93767a5645c99b26b4acd3600515', 'request_id': 'req-ec8bd076-68be-4446-b587-575e4d57c843'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:24.518695 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 848/1694] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:24 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:24.523333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ec8bd076-68be-4446-b587-575e4d57c843 admin admin] The token cache is being invalidate because project 587b93767a5645c99b26b4acd3600515 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:33:24.523815 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ec8bd076-68be-4446-b587-575e4d57c843 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:24.525802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ec8bd076-68be-4446-b587-575e4d57c843 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:24.530088 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116611a5-038d-4ccf-bffc-178b9892deee None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:24.531580 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-116611a5-038d-4ccf-bffc-178b9892deee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:24.532357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 847/1695] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:33:24 2022] DELETE /identity/v3/projects/587b93767a5645c99b26b4acd3600515 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:24.536821 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18522e01-394a-44b4-a8e9-8c3744a83083 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:24.537231 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18522e01-394a-44b4-a8e9-8c3744a83083 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:24.538371 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18522e01-394a-44b4-a8e9-8c3744a83083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:24.553599 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:24.589475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-18522e01-394a-44b4-a8e9-8c3744a83083 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:24.593611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-18522e01-394a-44b4-a8e9-8c3744a83083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:24.658272 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116611a5-038d-4ccf-bffc-178b9892deee None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:24.774395 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0c25b-0d18-4d24-91c1-a2f2cc7c90d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:24.776368 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0c25b-0d18-4d24-91c1-a2f2cc7c90d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:24.776368 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0c25b-0d18-4d24-91c1-a2f2cc7c90d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:24.776818 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0c25b-0d18-4d24-91c1-a2f2cc7c90d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:24.777486 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 848/1696] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:24.783304 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-57e0c25b-0d18-4d24-91c1-a2f2cc7c90d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:24.789787 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b38354-8627-4685-afcf-df7cdb98ea2d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:24.790114 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b38354-8627-4685-afcf-df7cdb98ea2d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:24.790391 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b38354-8627-4685-afcf-df7cdb98ea2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:24.853064 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-48b38354-8627-4685-afcf-df7cdb98ea2d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:24.854286 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-48b38354-8627-4685-afcf-df7cdb98ea2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:24.886107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 849/1697] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:24.890588 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b38354-8627-4685-afcf-df7cdb98ea2d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:24.921314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b38354-8627-4685-afcf-df7cdb98ea2d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:24.940682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737183b-9d8d-4641-a3ed-3d1b9b347590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:24.941726 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737183b-9d8d-4641-a3ed-3d1b9b347590 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:24.942197 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737183b-9d8d-4641-a3ed-3d1b9b347590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:24.942682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737183b-9d8d-4641-a3ed-3d1b9b347590 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:24.943616 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737183b-9d8d-4641-a3ed-3d1b9b347590 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:33:24.944068 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737183b-9d8d-4641-a3ed-3d1b9b347590 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:24.948083 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737183b-9d8d-4641-a3ed-3d1b9b347590 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:24.960325 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0c25b-0d18-4d24-91c1-a2f2cc7c90d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:24.962727 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0c25b-0d18-4d24-91c1-a2f2cc7c90d0 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:24.981893 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 850/1698] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:33:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:33:24.986068 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737183b-9d8d-4641-a3ed-3d1b9b347590 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:24.987513 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 849/1699] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:24 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 12:33:25.007199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737183b-9d8d-4641-a3ed-3d1b9b347590 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:25.024374 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1871d427-f1fb-469c-b4b9-ac01b084736a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:25.025308 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871d427-f1fb-469c-b4b9-ac01b084736a admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:25.025738 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871d427-f1fb-469c-b4b9-ac01b084736a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:25.026244 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1871d427-f1fb-469c-b4b9-ac01b084736a admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:25.027703 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1871d427-f1fb-469c-b4b9-ac01b084736a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:25.031943 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1871d427-f1fb-469c-b4b9-ac01b084736a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:25.051196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:25.090951 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 851/1700] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:24 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:33:25.098080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0c25b-0d18-4d24-91c1-a2f2cc7c90d0 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:25.130846 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0c25b-0d18-4d24-91c1-a2f2cc7c90d0 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:25.285519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a274b6-d2b1-47c3-934f-c2d06cdbc030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:25.286839 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a274b6-d2b1-47c3-934f-c2d06cdbc030 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:25.287281 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a274b6-d2b1-47c3-934f-c2d06cdbc030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:25.289496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a274b6-d2b1-47c3-934f-c2d06cdbc030 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:25.290336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a274b6-d2b1-47c3-934f-c2d06cdbc030 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:25.291110 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1871d427-f1fb-469c-b4b9-ac01b084736a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:25.292740 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a274b6-d2b1-47c3-934f-c2d06cdbc030 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:25.294612 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1871d427-f1fb-469c-b4b9-ac01b084736a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:25.315936 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1871d427-f1fb-469c-b4b9-ac01b084736a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:25.330659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ea975-f94c-4e73-aeb6-9644873d45d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:25.331670 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ea975-f94c-4e73-aeb6-9644873d45d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:25.331965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ea975-f94c-4e73-aeb6-9644873d45d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:25.332255 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ea975-f94c-4e73-aeb6-9644873d45d1 admin admin] PATH_INFO: `/v3/users/8f8fcb31e8a14dadb3a9a77d670fe6cc` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:25.334780 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 850/1701] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:25 2022] POST /identity/v3/users => generated 288 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:25.336054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194ea975-f94c-4e73-aeb6-9644873d45d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f8fcb31e8a14dadb3a9a77d670fe6cc)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:25.338684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a274b6-d2b1-47c3-934f-c2d06cdbc030 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:25.340563 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194ea975-f94c-4e73-aeb6-9644873d45d1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:25.356314 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a274b6-d2b1-47c3-934f-c2d06cdbc030 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:25.378747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2fa377-8042-4279-8224-2eb23fcbe59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:25.380504 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2fa377-8042-4279-8224-2eb23fcbe59d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:25.381007 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2fa377-8042-4279-8224-2eb23fcbe59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:25.381412 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2fa377-8042-4279-8224-2eb23fcbe59d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:25.382268 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2fa377-8042-4279-8224-2eb23fcbe59d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:25.384050 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2fa377-8042-4279-8224-2eb23fcbe59d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:25.390152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:25.421603 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 851/1702] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:33:25.426070 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2fa377-8042-4279-8224-2eb23fcbe59d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:25.453041 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2fa377-8042-4279-8224-2eb23fcbe59d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:25.467149 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-194ea975-f94c-4e73-aeb6-9644873d45d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f8fcb31e8a14dadb3a9a77d670fe6cc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:25.483873 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628002f-04d3-48c8-a395-af6cab0558a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:25.485000 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-194ea975-f94c-4e73-aeb6-9644873d45d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f8fcb31e8a14dadb3a9a77d670fe6cc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:25.485288 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628002f-04d3-48c8-a395-af6cab0558a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:25.485788 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628002f-04d3-48c8-a395-af6cab0558a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:25.486239 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628002f-04d3-48c8-a395-af6cab0558a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:25.489108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0628002f-04d3-48c8-a395-af6cab0558a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:25.492925 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0628002f-04d3-48c8-a395-af6cab0558a1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:25.515352 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-194ea975-f94c-4e73-aeb6-9644873d45d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f8fcb31e8a14dadb3a9a77d670fe6cc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:25.517000 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-194ea975-f94c-4e73-aeb6-9644873d45d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f8fcb31e8a14dadb3a9a77d670fe6cc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:25.518876 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-194ea975-f94c-4e73-aeb6-9644873d45d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f8fcb31e8a14dadb3a9a77d670fe6cc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:25.534213 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 852/1703] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:25 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:33:25.536247 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628002f-04d3-48c8-a395-af6cab0558a1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:25.543809 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0628002f-04d3-48c8-a395-af6cab0558a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:25.550287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 852/1704] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:33:25 2022] DELETE /identity/v3/users/8f8fcb31e8a14dadb3a9a77d670fe6cc => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:25.556852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687bced-c6d7-42f4-82a8-c2aef41399ac None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:25.557191 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687bced-c6d7-42f4-82a8-c2aef41399ac None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:25.557489 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687bced-c6d7-42f4-82a8-c2aef41399ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:25.580553 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628002f-04d3-48c8-a395-af6cab0558a1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:25.604225 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6687bced-c6d7-42f4-82a8-c2aef41399ac None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:25.605188 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6687bced-c6d7-42f4-82a8-c2aef41399ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:25.756200 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 853/1705] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:25.769952 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9efc67-429e-44e1-a2f8-c7c7b0d7a2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:25.771674 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9efc67-429e-44e1-a2f8-c7c7b0d7a2d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:25.772089 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9efc67-429e-44e1-a2f8-c7c7b0d7a2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:25.772474 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9efc67-429e-44e1-a2f8-c7c7b0d7a2d4 admin admin] PATH_INFO: `/v3/projects/f3394a1d2a42406b89c7b100c4809585/users/f339584f5ea64315a3a664b9352552cc/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:25.815091 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9efc67-429e-44e1-a2f8-c7c7b0d7a2d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3394a1d2a42406b89c7b100c4809585, user_id=f339584f5ea64315a3a664b9352552cc, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:25.821650 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9efc67-429e-44e1-a2f8-c7c7b0d7a2d4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:25.871891 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687bced-c6d7-42f4-82a8-c2aef41399ac None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:25.872925 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 853/1706] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:25 2022] PUT /identity/v3/projects/f3394a1d2a42406b89c7b100c4809585/users/f339584f5ea64315a3a664b9352552cc/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:25.875286 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67ccca-7abe-4b7d-a903-30c8b49b52a1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:25.875786 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67ccca-7abe-4b7d-a903-30c8b49b52a1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:25.876313 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67ccca-7abe-4b7d-a903-30c8b49b52a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:25.879015 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6687bced-c6d7-42f4-82a8-c2aef41399ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:25.934636 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4f67ccca-7abe-4b7d-a903-30c8b49b52a1 None None] MFA Rules not processed for user `f339584f5ea64315a3a664b9352552cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:25.939555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4f67ccca-7abe-4b7d-a903-30c8b49b52a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:26.033101 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687bced-c6d7-42f4-82a8-c2aef41399ac None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:26.069467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:26.109742 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 854/1707] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:26.113371 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e78484-6457-4fc2-bd3a-7f0e42aba1ab None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:26.115221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e78484-6457-4fc2-bd3a-7f0e42aba1ab None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:26.115655 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e78484-6457-4fc2-bd3a-7f0e42aba1ab None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:26.119856 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 855/1708] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jun 29 12:33:26 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:33:26.120885 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e78484-6457-4fc2-bd3a-7f0e42aba1ab None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:26.152055 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a34a87-a22f-4bd9-a529-f46296edca6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:26.152648 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a34a87-a22f-4bd9-a529-f46296edca6d service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:26.153511 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a34a87-a22f-4bd9-a529-f46296edca6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:26.153976 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a34a87-a22f-4bd9-a529-f46296edca6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:26.202916 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:26.219003 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e78484-6457-4fc2-bd3a-7f0e42aba1ab None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:26.229761 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b9fc92-507e-4aec-9402-074f664c2036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:26.231037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9fc92-507e-4aec-9402-074f664c2036 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:26.231615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9fc92-507e-4aec-9402-074f664c2036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:26.232448 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b9fc92-507e-4aec-9402-074f664c2036 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:26.233663 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b9fc92-507e-4aec-9402-074f664c2036 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:26.236633 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b9fc92-507e-4aec-9402-074f664c2036 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:26.283659 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 856/1709] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:26 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:26.286490 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede0135-00b9-427d-bf26-eead3de5cb32 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:26.287664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede0135-00b9-427d-bf26-eead3de5cb32 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:26.288143 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede0135-00b9-427d-bf26-eead3de5cb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:26.310838 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a34a87-a22f-4bd9-a529-f46296edca6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:26.312371 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a34a87-a22f-4bd9-a529-f46296edca6d service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:26.313024 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3ede0135-00b9-427d-bf26-eead3de5cb32 None None] MFA Rules not processed for user `c98135cacb9d4f22846ad33500b9bd1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:26.315111 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3ede0135-00b9-427d-bf26-eead3de5cb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:26.338834 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 854/1710] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:26.339728 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a34a87-a22f-4bd9-a529-f46296edca6d service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:26.372521 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a34a87-a22f-4bd9-a529-f46296edca6d service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:26.407008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 857/1711] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jun 29 12:33:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:26.472080 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5c334-f6ee-4a1e-a1da-91717452584d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:26.473011 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5c334-f6ee-4a1e-a1da-91717452584d admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:26.473859 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5c334-f6ee-4a1e-a1da-91717452584d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:26.474359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d5c334-f6ee-4a1e-a1da-91717452584d admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:26.475152 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d5c334-f6ee-4a1e-a1da-91717452584d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:26.477384 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d5c334-f6ee-4a1e-a1da-91717452584d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:26.524105 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 855/1712] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:26 2022] POST /identity/v3/users => generated 295 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:26.525502 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede0135-00b9-427d-bf26-eead3de5cb32 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:26.550511 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede0135-00b9-427d-bf26-eead3de5cb32 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:26.556647 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57551dc1-cc0d-4235-b813-e14e209b9834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:26.557625 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57551dc1-cc0d-4235-b813-e14e209b9834 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:26.558025 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57551dc1-cc0d-4235-b813-e14e209b9834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:26.558682 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57551dc1-cc0d-4235-b813-e14e209b9834 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:26.559539 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57551dc1-cc0d-4235-b813-e14e209b9834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:26.561581 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57551dc1-cc0d-4235-b813-e14e209b9834 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:26.577887 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 858/1713] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:26.579755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5c334-f6ee-4a1e-a1da-91717452584d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:26.594435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d5c334-f6ee-4a1e-a1da-91717452584d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:26.600459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca47016-2afa-440c-a98a-03afbdac6bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:26.603880 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca47016-2afa-440c-a98a-03afbdac6bd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:26.604529 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca47016-2afa-440c-a98a-03afbdac6bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:26.604941 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca47016-2afa-440c-a98a-03afbdac6bd7 admin admin] PATH_INFO: `/v3/projects/553daa8dbd9f46c68105ec7830f2ee0b/users/96af918c89e74cc2947711dec19e4b4f/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:26.640962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca47016-2afa-440c-a98a-03afbdac6bd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=553daa8dbd9f46c68105ec7830f2ee0b, user_id=96af918c89e74cc2947711dec19e4b4f, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:26.658634 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca47016-2afa-440c-a98a-03afbdac6bd7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:26.693863 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 856/1714] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:26 2022] PUT /identity/v3/projects/553daa8dbd9f46c68105ec7830f2ee0b/users/96af918c89e74cc2947711dec19e4b4f/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:33:26.695731 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57551dc1-cc0d-4235-b813-e14e209b9834 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:26.709755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57551dc1-cc0d-4235-b813-e14e209b9834 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:26.856863 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7eb70-6468-4c2a-b4df-f7ffc2a39122 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:26.857155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7eb70-6468-4c2a-b4df-f7ffc2a39122 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:26.857445 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff7eb70-6468-4c2a-b4df-f7ffc2a39122 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:26.859179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 857/1715] 10.222.0.52 () {54 vars in 960 bytes} [Wed Jun 29 12:33:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 29 12:33:26.867572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7de84-70a6-4140-bf75-215afde8aaa0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:26.867928 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7de84-70a6-4140-bf75-215afde8aaa0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:26.868214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7de84-70a6-4140-bf75-215afde8aaa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:26.953368 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b06f1-4059-41c6-a459-5ebd288bdd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:26.954076 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b06f1-4059-41c6-a459-5ebd288bdd37 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:26.954366 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b06f1-4059-41c6-a459-5ebd288bdd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:26.954681 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b06f1-4059-41c6-a459-5ebd288bdd37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:26.958966 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3ef7de84-70a6-4140-bf75-215afde8aaa0 None None] MFA Rules not processed for user `c98135cacb9d4f22846ad33500b9bd1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:26.960243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3ef7de84-70a6-4140-bf75-215afde8aaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:26.962906 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b06f1-4059-41c6-a459-5ebd288bdd37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:26.966779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b06f1-4059-41c6-a459-5ebd288bdd37 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:27.002821 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 859/1716] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:27.006356 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b06f1-4059-41c6-a459-5ebd288bdd37 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:27.044791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b06f1-4059-41c6-a459-5ebd288bdd37 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:27.069383 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3805ff56-1ea9-4e39-a065-e5fd04f9b2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:27.070740 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805ff56-1ea9-4e39-a065-e5fd04f9b2af admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:27.071239 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805ff56-1ea9-4e39-a065-e5fd04f9b2af admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:27.072492 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3805ff56-1ea9-4e39-a065-e5fd04f9b2af admin admin] PATH_INFO: `/v3/projects/553daa8dbd9f46c68105ec7830f2ee0b/users/96af918c89e74cc2947711dec19e4b4f/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:27.081501 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3805ff56-1ea9-4e39-a065-e5fd04f9b2af admin admin] RBAC: Authorizing `identity:create_grant(project_id=553daa8dbd9f46c68105ec7830f2ee0b, user_id=96af918c89e74cc2947711dec19e4b4f, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:27.082875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:27.089752 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3805ff56-1ea9-4e39-a065-e5fd04f9b2af admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:27.133116 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 860/1717] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:27 2022] PUT /identity/v3/projects/553daa8dbd9f46c68105ec7830f2ee0b/users/96af918c89e74cc2947711dec19e4b4f/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:27.135493 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc12a4-ae34-4da5-9b58-ce6a8aa85d8e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:27.135902 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc12a4-ae34-4da5-9b58-ce6a8aa85d8e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:27.136292 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc12a4-ae34-4da5-9b58-ce6a8aa85d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:27.187898 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 858/1718] 10.222.0.52 () {58 vars in 1057 bytes} [Wed Jun 29 12:33:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:27.202752 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7de84-70a6-4140-bf75-215afde8aaa0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:27.217192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7de84-70a6-4140-bf75-215afde8aaa0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:27.229343 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a7bc12a4-ae34-4da5-9b58-ce6a8aa85d8e None None] MFA Rules not processed for user `96af918c89e74cc2947711dec19e4b4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:27.231100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a7bc12a4-ae34-4da5-9b58-ce6a8aa85d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:27.315245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:27.384437 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 861/1719] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:27 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:27.389640 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68973c8-2e45-4c51-8819-53b7b3b2f365 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98135cacb9d4f22846ad33500b9bd1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:27.390600 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68973c8-2e45-4c51-8819-53b7b3b2f365 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:27.390896 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68973c8-2e45-4c51-8819-53b7b3b2f365 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:27.391406 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68973c8-2e45-4c51-8819-53b7b3b2f365 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:27.393258 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc12a4-ae34-4da5-9b58-ce6a8aa85d8e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:27.395707 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a7bc12a4-ae34-4da5-9b58-ce6a8aa85d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:27.412109 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68973c8-2e45-4c51-8819-53b7b3b2f365 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:27.413776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68973c8-2e45-4c51-8819-53b7b3b2f365 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:27.437980 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 859/1720] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 12:33:27.620186 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc12a4-ae34-4da5-9b58-ce6a8aa85d8e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:27.626233 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c5d02-9d87-40ca-bfb3-b1a5aca46072 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:27.627586 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c5d02-9d87-40ca-bfb3-b1a5aca46072 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:27.628088 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c5d02-9d87-40ca-bfb3-b1a5aca46072 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:27.628588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c5d02-9d87-40ca-bfb3-b1a5aca46072 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:27.781954 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c5d02-9d87-40ca-bfb3-b1a5aca46072 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:27.783677 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c5d02-9d87-40ca-bfb3-b1a5aca46072 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:27.798563 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 862/1721] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:28.858141 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68973c8-2e45-4c51-8819-53b7b3b2f365 service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:28.858141 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c68973c8-2e45-4c51-8819-53b7b3b2f365 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:29.076706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68973c8-2e45-4c51-8819-53b7b3b2f365 service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:29.102560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633db1a3-8f5e-46e4-985e-366296547e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:29.105279 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633db1a3-8f5e-46e4-985e-366296547e23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:29.105840 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633db1a3-8f5e-46e4-985e-366296547e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:29.106962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633db1a3-8f5e-46e4-985e-366296547e23 admin admin] PATH_INFO: `/v3/projects/c9758340629f4e159b3a2fc279a76b98` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:29.110966 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633db1a3-8f5e-46e4-985e-366296547e23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9758340629f4e159b3a2fc279a76b98)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:29.116392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633db1a3-8f5e-46e4-985e-366296547e23 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:29.229766 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-633db1a3-8f5e-46e4-985e-366296547e23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9758340629f4e159b3a2fc279a76b98', 'request_id': 'req-633db1a3-8f5e-46e4-985e-366296547e23'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:29.243735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-633db1a3-8f5e-46e4-985e-366296547e23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9758340629f4e159b3a2fc279a76b98', 'request_id': 'req-633db1a3-8f5e-46e4-985e-366296547e23'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:29.251558 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:29.253607 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-633db1a3-8f5e-46e4-985e-366296547e23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9758340629f4e159b3a2fc279a76b98', 'request_id': 'req-633db1a3-8f5e-46e4-985e-366296547e23'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:29.288059 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-633db1a3-8f5e-46e4-985e-366296547e23 admin admin] The token cache is being invalidate because project c9758340629f4e159b3a2fc279a76b98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:33:29.289515 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-633db1a3-8f5e-46e4-985e-366296547e23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:29.290655 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-633db1a3-8f5e-46e4-985e-366296547e23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:29.293618 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 860/1722] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:33:28 2022] DELETE /identity/v3/projects/c9758340629f4e159b3a2fc279a76b98 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:29.298259 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c5d02-9d87-40ca-bfb3-b1a5aca46072 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:29.325403 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c5d02-9d87-40ca-bfb3-b1a5aca46072 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:29.469741 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2abb4e6-0efa-4846-ad4f-f5c174088145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:29.470678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abb4e6-0efa-4846-ad4f-f5c174088145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:29.470973 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abb4e6-0efa-4846-ad4f-f5c174088145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:29.471256 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2abb4e6-0efa-4846-ad4f-f5c174088145 admin admin] PATH_INFO: `/v3/users/0ecc669e82b349168b87be347b50ba6a` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:29.473069 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abb4e6-0efa-4846-ad4f-f5c174088145 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ecc669e82b349168b87be347b50ba6a)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:29.478295 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2abb4e6-0efa-4846-ad4f-f5c174088145 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:29.603638 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:29.638535 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d2abb4e6-0efa-4846-ad4f-f5c174088145 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ecc669e82b349168b87be347b50ba6a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:29.639516 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d2abb4e6-0efa-4846-ad4f-f5c174088145 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ecc669e82b349168b87be347b50ba6a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:29.702601 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d2abb4e6-0efa-4846-ad4f-f5c174088145 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ecc669e82b349168b87be347b50ba6a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:29.705741 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d2abb4e6-0efa-4846-ad4f-f5c174088145 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ecc669e82b349168b87be347b50ba6a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:29.729796 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d2abb4e6-0efa-4846-ad4f-f5c174088145 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ecc669e82b349168b87be347b50ba6a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:29.764086 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 863/1723] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:33:29 2022] DELETE /identity/v3/users/0ecc669e82b349168b87be347b50ba6a => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:30.550665 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633db1a3-8f5e-46e4-985e-366296547e23 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:30.553464 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-633db1a3-8f5e-46e4-985e-366296547e23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:30.581251 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633db1a3-8f5e-46e4-985e-366296547e23 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:30.711685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15596be-8d39-4095-adec-b204667b5126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:30.712590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15596be-8d39-4095-adec-b204667b5126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:30.712878 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15596be-8d39-4095-adec-b204667b5126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:30.713163 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15596be-8d39-4095-adec-b204667b5126 admin admin] PATH_INFO: `/v3/users/558836b50d72443d97d97597b5574954` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:30.719315 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15596be-8d39-4095-adec-b204667b5126 admin admin] RBAC: Authorizing `identity:delete_user(user_id=558836b50d72443d97d97597b5574954)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:30.721301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15596be-8d39-4095-adec-b204667b5126 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:30.815982 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3d9c1-5c6c-49de-becf-eb410dec3395 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:30.816538 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3d9c1-5c6c-49de-becf-eb410dec3395 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:30.817742 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3d9c1-5c6c-49de-becf-eb410dec3395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:30.831425 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f15596be-8d39-4095-adec-b204667b5126 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '558836b50d72443d97d97597b5574954'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:30.855971 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f15596be-8d39-4095-adec-b204667b5126 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '558836b50d72443d97d97597b5574954'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:30.870378 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-00c3d9c1-5c6c-49de-becf-eb410dec3395 None None] MFA Rules not processed for user `8a232c3292ec4a0abda3a8bc91c5a267`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:30.872589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-00c3d9c1-5c6c-49de-becf-eb410dec3395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:30.885299 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f15596be-8d39-4095-adec-b204667b5126 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '558836b50d72443d97d97597b5574954'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:30.888134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f15596be-8d39-4095-adec-b204667b5126 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '558836b50d72443d97d97597b5574954'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:30.889196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f15596be-8d39-4095-adec-b204667b5126 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '558836b50d72443d97d97597b5574954'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:30.915357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 861/1724] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:33:30 2022] DELETE /identity/v3/users/558836b50d72443d97d97597b5574954 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:30.951889 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21649b5c-67b2-4d15-962a-8b13af83ee5b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:30.952188 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21649b5c-67b2-4d15-962a-8b13af83ee5b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:30.952479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21649b5c-67b2-4d15-962a-8b13af83ee5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:31.015450 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-21649b5c-67b2-4d15-962a-8b13af83ee5b None None] MFA Rules not processed for user `350a57ee307a4419974dcecf37e466e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:31.016900 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-21649b5c-67b2-4d15-962a-8b13af83ee5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:31.040916 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 864/1725] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:33:31.069401 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302e0f7-1b01-4fca-9fa4-55148014957d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:31.070010 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302e0f7-1b01-4fca-9fa4-55148014957d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:31.070497 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5302e0f7-1b01-4fca-9fa4-55148014957d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:31.106167 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5302e0f7-1b01-4fca-9fa4-55148014957d None None] MFA Rules not processed for user `8a232c3292ec4a0abda3a8bc91c5a267`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:31.107703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5302e0f7-1b01-4fca-9fa4-55148014957d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:31.185099 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:31.202675 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 862/1726] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:31.208028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 865/1727] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:31.212824 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5b360-3c44-4a67-8a4b-4ae4c71bdcd6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:31.213191 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5b360-3c44-4a67-8a4b-4ae4c71bdcd6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:31.213554 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5b360-3c44-4a67-8a4b-4ae4c71bdcd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:31.217617 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302e0f7-1b01-4fca-9fa4-55148014957d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:31.223013 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5302e0f7-1b01-4fca-9fa4-55148014957d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:31.241379 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0bc5b360-3c44-4a67-8a4b-4ae4c71bdcd6 None None] MFA Rules not processed for user `350a57ee307a4419974dcecf37e466e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:31.242998 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0bc5b360-3c44-4a67-8a4b-4ae4c71bdcd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:31.276658 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 863/1728] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:31.282984 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5b360-3c44-4a67-8a4b-4ae4c71bdcd6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:31.288090 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0bc5b360-3c44-4a67-8a4b-4ae4c71bdcd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:31.313638 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:31.370576 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5302e0f7-1b01-4fca-9fa4-55148014957d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:31.381591 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb1d00-26ef-4345-bc52-8ec6064b4b8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:31.382723 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb1d00-26ef-4345-bc52-8ec6064b4b8c service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:31.383490 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb1d00-26ef-4345-bc52-8ec6064b4b8c service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:31.383490 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb1d00-26ef-4345-bc52-8ec6064b4b8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:31.399715 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bb1d00-26ef-4345-bc52-8ec6064b4b8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:31.401667 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bb1d00-26ef-4345-bc52-8ec6064b4b8c service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:31.417256 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 866/1729] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 12:33:31.420352 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb1d00-26ef-4345-bc52-8ec6064b4b8c service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:31.424943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a0bb1d00-26ef-4345-bc52-8ec6064b4b8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:31.536093 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb1d00-26ef-4345-bc52-8ec6064b4b8c service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:31.546852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc5b360-3c44-4a67-8a4b-4ae4c71bdcd6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:31.551247 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbdbd31-4ca9-4683-81c7-8ffe6a917b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:31.552463 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdbd31-4ca9-4683-81c7-8ffe6a917b9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:31.552866 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdbd31-4ca9-4683-81c7-8ffe6a917b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:31.553244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbdbd31-4ca9-4683-81c7-8ffe6a917b9c admin admin] PATH_INFO: `/v3/projects/0ecbd8a904ce455bb6d0a71db1e6edac` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:31.555381 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbdbd31-4ca9-4683-81c7-8ffe6a917b9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ecbd8a904ce455bb6d0a71db1e6edac)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:31.556674 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-4bbdbd31-4ca9-4683-81c7-8ffe6a917b9c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 29 12:33:31.556674 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jun 29 12:33:31.556674 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: Jun 29 12:33:31.558897 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbdbd31-4ca9-4683-81c7-8ffe6a917b9c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:31.561630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b272867-4cef-4b55-bdb4-ad5726e70bfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:31.562858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b272867-4cef-4b55-bdb4-ad5726e70bfa service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:31.563957 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b272867-4cef-4b55-bdb4-ad5726e70bfa service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:31.564245 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b272867-4cef-4b55-bdb4-ad5726e70bfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:31.610666 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b272867-4cef-4b55-bdb4-ad5726e70bfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:31.625873 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b272867-4cef-4b55-bdb4-ad5726e70bfa service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:31.668705 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 864/1730] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:31 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 12:33:31.671890 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1eb9cc-94b6-4731-9738-93e5d910d38e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:31.672181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1eb9cc-94b6-4731-9738-93e5d910d38e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:31.672465 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1eb9cc-94b6-4731-9738-93e5d910d38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:31.735924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ef1eb9cc-94b6-4731-9738-93e5d910d38e None None] MFA Rules not processed for user `6252d0519ed9421e89710ca5294524c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:31.737166 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ef1eb9cc-94b6-4731-9738-93e5d910d38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:31.743867 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4bbdbd31-4ca9-4683-81c7-8ffe6a917b9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ecbd8a904ce455bb6d0a71db1e6edac', 'request_id': 'req-4bbdbd31-4ca9-4683-81c7-8ffe6a917b9c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:31.757979 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4bbdbd31-4ca9-4683-81c7-8ffe6a917b9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ecbd8a904ce455bb6d0a71db1e6edac', 'request_id': 'req-4bbdbd31-4ca9-4683-81c7-8ffe6a917b9c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:31.812377 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4bbdbd31-4ca9-4683-81c7-8ffe6a917b9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ecbd8a904ce455bb6d0a71db1e6edac', 'request_id': 'req-4bbdbd31-4ca9-4683-81c7-8ffe6a917b9c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:31.865468 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4bbdbd31-4ca9-4683-81c7-8ffe6a917b9c admin admin] The token cache is being invalidate because project 0ecbd8a904ce455bb6d0a71db1e6edac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:33:31.866087 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4bbdbd31-4ca9-4683-81c7-8ffe6a917b9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:31.870414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4bbdbd31-4ca9-4683-81c7-8ffe6a917b9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:31.876371 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 867/1731] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:33:31 2022] DELETE /identity/v3/projects/0ecbd8a904ce455bb6d0a71db1e6edac => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:31.878981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c715f60-5b6c-4dd9-b30c-10531f531cc6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:31.880265 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c715f60-5b6c-4dd9-b30c-10531f531cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:31.880674 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c715f60-5b6c-4dd9-b30c-10531f531cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:31.912220 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 865/1732] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:31.912541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3c715f60-5b6c-4dd9-b30c-10531f531cc6 None None] MFA Rules not processed for user `a4a30b65a91a4fd89e8b59df670f60ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:31.913361 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1eb9cc-94b6-4731-9738-93e5d910d38e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:31.914071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3c715f60-5b6c-4dd9-b30c-10531f531cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:31.914871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ef1eb9cc-94b6-4731-9738-93e5d910d38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:31.976640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:32.000348 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 868/1733] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:32.001610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c715f60-5b6c-4dd9-b30c-10531f531cc6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:32.009254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3c715f60-5b6c-4dd9-b30c-10531f531cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:32.073929 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1eb9cc-94b6-4731-9738-93e5d910d38e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:32.078853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817bab5-831c-45e5-b096-ad0ed179830a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:32.079743 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817bab5-831c-45e5-b096-ad0ed179830a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:32.080234 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817bab5-831c-45e5-b096-ad0ed179830a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:32.080514 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817bab5-831c-45e5-b096-ad0ed179830a admin admin] PATH_INFO: `/v3/users/669758a928794a86a45afe6ad3d78595` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:32.081991 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817bab5-831c-45e5-b096-ad0ed179830a admin admin] RBAC: Authorizing `identity:delete_user(user_id=669758a928794a86a45afe6ad3d78595)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:32.085027 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817bab5-831c-45e5-b096-ad0ed179830a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:32.101612 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c715f60-5b6c-4dd9-b30c-10531f531cc6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:32.107436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f472c3-3340-4d7f-844c-d56c1c6a6e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:32.108904 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f472c3-3340-4d7f-844c-d56c1c6a6e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:32.109658 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f472c3-3340-4d7f-844c-d56c1c6a6e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:32.110058 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f472c3-3340-4d7f-844c-d56c1c6a6e68 admin admin] PATH_INFO: `/v3/users/9a58575aafa8411f9c6f843d4c176896` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:32.111726 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f472c3-3340-4d7f-844c-d56c1c6a6e68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a58575aafa8411f9c6f843d4c176896)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:32.116061 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f472c3-3340-4d7f-844c-d56c1c6a6e68 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:32.126041 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:32.206283 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2817bab5-831c-45e5-b096-ad0ed179830a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '669758a928794a86a45afe6ad3d78595'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:32.217162 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c3f472c3-3340-4d7f-844c-d56c1c6a6e68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a58575aafa8411f9c6f843d4c176896'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:32.219121 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c3f472c3-3340-4d7f-844c-d56c1c6a6e68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a58575aafa8411f9c6f843d4c176896'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:32.236495 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2817bab5-831c-45e5-b096-ad0ed179830a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '669758a928794a86a45afe6ad3d78595'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:32.255022 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c3f472c3-3340-4d7f-844c-d56c1c6a6e68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a58575aafa8411f9c6f843d4c176896'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:32.256226 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c3f472c3-3340-4d7f-844c-d56c1c6a6e68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a58575aafa8411f9c6f843d4c176896'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:32.258757 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2817bab5-831c-45e5-b096-ad0ed179830a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '669758a928794a86a45afe6ad3d78595'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:32.259580 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2817bab5-831c-45e5-b096-ad0ed179830a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '669758a928794a86a45afe6ad3d78595'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:32.260522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2817bab5-831c-45e5-b096-ad0ed179830a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '669758a928794a86a45afe6ad3d78595'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:32.286929 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c3f472c3-3340-4d7f-844c-d56c1c6a6e68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a58575aafa8411f9c6f843d4c176896'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:32.309833 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 866/1734] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:33:31 2022] DELETE /identity/v3/users/669758a928794a86a45afe6ad3d78595 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:32.309833 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817bab5-831c-45e5-b096-ad0ed179830a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:32.313985 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2817bab5-831c-45e5-b096-ad0ed179830a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:32.325050 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 869/1735] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:33:32 2022] DELETE /identity/v3/users/9a58575aafa8411f9c6f843d4c176896 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:32.329939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2c209-96ed-4644-a5e9-e8d5a1ad68fd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:32.330231 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2c209-96ed-4644-a5e9-e8d5a1ad68fd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:32.330552 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2c209-96ed-4644-a5e9-e8d5a1ad68fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:32.399343 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e1c2c209-96ed-4644-a5e9-e8d5a1ad68fd None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:32.406549 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e1c2c209-96ed-4644-a5e9-e8d5a1ad68fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:32.492859 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817bab5-831c-45e5-b096-ad0ed179830a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:32.593234 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 870/1736] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:32.613244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6aeaf-e24f-4b57-a771-c7792fd595ae None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:32.616017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6aeaf-e24f-4b57-a771-c7792fd595ae None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:32.616375 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6aeaf-e24f-4b57-a771-c7792fd595ae None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:32.619268 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 871/1737] 10.222.0.52 () {54 vars in 912 bytes} [Wed Jun 29 12:33:32 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:33:32.622490 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6aeaf-e24f-4b57-a771-c7792fd595ae None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:32.626811 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5fd6aeaf-e24f-4b57-a771-c7792fd595ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:32.654075 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:32.658802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e5906e-d19d-4746-9fc1-f55f83c8f507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:32.663104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5906e-d19d-4746-9fc1-f55f83c8f507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:32.664164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5906e-d19d-4746-9fc1-f55f83c8f507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:32.664538 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e5906e-d19d-4746-9fc1-f55f83c8f507 admin admin] PATH_INFO: `/v3/projects/dda90d00caeb4656a8da048fb2039657` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:32.666936 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e5906e-d19d-4746-9fc1-f55f83c8f507 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dda90d00caeb4656a8da048fb2039657)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:32.670349 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e5906e-d19d-4746-9fc1-f55f83c8f507 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:32.746073 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-81e5906e-d19d-4746-9fc1-f55f83c8f507 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dda90d00caeb4656a8da048fb2039657', 'request_id': 'req-81e5906e-d19d-4746-9fc1-f55f83c8f507'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:32.747374 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:32.749578 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6aeaf-e24f-4b57-a771-c7792fd595ae None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:32.757733 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-81e5906e-d19d-4746-9fc1-f55f83c8f507 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dda90d00caeb4656a8da048fb2039657', 'request_id': 'req-81e5906e-d19d-4746-9fc1-f55f83c8f507'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:32.776033 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-81e5906e-d19d-4746-9fc1-f55f83c8f507 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dda90d00caeb4656a8da048fb2039657', 'request_id': 'req-81e5906e-d19d-4746-9fc1-f55f83c8f507'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:32.795445 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-81e5906e-d19d-4746-9fc1-f55f83c8f507 admin admin] The token cache is being invalidate because project dda90d00caeb4656a8da048fb2039657 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:33:32.795767 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-81e5906e-d19d-4746-9fc1-f55f83c8f507 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:32.796816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-81e5906e-d19d-4746-9fc1-f55f83c8f507 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:32.802485 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 867/1738] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:33:32 2022] DELETE /identity/v3/projects/dda90d00caeb4656a8da048fb2039657 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:32.804005 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46faf91-7451-4fa0-a55a-14a554fd6b21 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:32.804340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46faf91-7451-4fa0-a55a-14a554fd6b21 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:32.804990 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46faf91-7451-4fa0-a55a-14a554fd6b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:32.858446 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425be64-67b1-4c44-b344-85cec658a54b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:32.859236 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425be64-67b1-4c44-b344-85cec658a54b service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:32.859588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425be64-67b1-4c44-b344-85cec658a54b service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:32.859819 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425be64-67b1-4c44-b344-85cec658a54b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:32.864222 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4425be64-67b1-4c44-b344-85cec658a54b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:32.878562 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e46faf91-7451-4fa0-a55a-14a554fd6b21 None None] MFA Rules not processed for user `ce922cdbf2d34f10b6e49631f0c6d770`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:32.880936 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e46faf91-7451-4fa0-a55a-14a554fd6b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:32.902630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 868/1739] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jun 29 12:33:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:32.904209 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85273a84-5fe8-4b51-a964-5c1f83bda723 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:32.904505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85273a84-5fe8-4b51-a964-5c1f83bda723 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:32.904785 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85273a84-5fe8-4b51-a964-5c1f83bda723 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:32.906316 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 869/1740] 10.222.0.52 () {54 vars in 912 bytes} [Wed Jun 29 12:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:33:32.907618 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b26110-7c22-46bd-97a6-2486509d154d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:32.907906 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b26110-7c22-46bd-97a6-2486509d154d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:32.908189 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b26110-7c22-46bd-97a6-2486509d154d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:32.939321 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d4b26110-7c22-46bd-97a6-2486509d154d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:32.940546 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d4b26110-7c22-46bd-97a6-2486509d154d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:32.978742 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4425be64-67b1-4c44-b344-85cec658a54b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:32.980840 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4425be64-67b1-4c44-b344-85cec658a54b service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:32.998119 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 872/1741] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:32.999108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425be64-67b1-4c44-b344-85cec658a54b service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:33.000587 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4425be64-67b1-4c44-b344-85cec658a54b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:33.020803 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425be64-67b1-4c44-b344-85cec658a54b service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:33.030518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42f898e-bc4a-476c-a78f-1e6ee40b0f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:33.031355 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f898e-bc4a-476c-a78f-1e6ee40b0f39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:33.031646 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f898e-bc4a-476c-a78f-1e6ee40b0f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:33.031928 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42f898e-bc4a-476c-a78f-1e6ee40b0f39 admin admin] PATH_INFO: `/v3/projects/f08b62f392d0466892d2662cf4949261` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:33.033162 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f898e-bc4a-476c-a78f-1e6ee40b0f39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f08b62f392d0466892d2662cf4949261)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:33.038358 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42f898e-bc4a-476c-a78f-1e6ee40b0f39 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:33.078467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 870/1742] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:33.078467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b26110-7c22-46bd-97a6-2486509d154d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:33.080865 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d4b26110-7c22-46bd-97a6-2486509d154d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:33.108902 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b26110-7c22-46bd-97a6-2486509d154d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:33.120457 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e42f898e-bc4a-476c-a78f-1e6ee40b0f39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f08b62f392d0466892d2662cf4949261', 'request_id': 'req-e42f898e-bc4a-476c-a78f-1e6ee40b0f39'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:33.133597 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e42f898e-bc4a-476c-a78f-1e6ee40b0f39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f08b62f392d0466892d2662cf4949261', 'request_id': 'req-e42f898e-bc4a-476c-a78f-1e6ee40b0f39'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:33.147123 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e42f898e-bc4a-476c-a78f-1e6ee40b0f39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f08b62f392d0466892d2662cf4949261', 'request_id': 'req-e42f898e-bc4a-476c-a78f-1e6ee40b0f39'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:33.157772 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:33.161641 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e42f898e-bc4a-476c-a78f-1e6ee40b0f39 admin admin] The token cache is being invalidate because project f08b62f392d0466892d2662cf4949261 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:33:33.162027 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e42f898e-bc4a-476c-a78f-1e6ee40b0f39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:33.163328 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e42f898e-bc4a-476c-a78f-1e6ee40b0f39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:33.166220 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 873/1743] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:33:32 2022] DELETE /identity/v3/projects/f08b62f392d0466892d2662cf4949261 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:33.168175 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc33c57-9dc3-433f-b2eb-8647f8f304ef None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:33.168605 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc33c57-9dc3-433f-b2eb-8647f8f304ef None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:33.168987 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc33c57-9dc3-433f-b2eb-8647f8f304ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:33.203988 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dbc33c57-9dc3-433f-b2eb-8647f8f304ef None None] MFA Rules not processed for user `ce922cdbf2d34f10b6e49631f0c6d770`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:33.205913 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dbc33c57-9dc3-433f-b2eb-8647f8f304ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:33.246488 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef56d6-ca6b-481a-a960-5f46ec6022ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:33.248238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef56d6-ca6b-481a-a960-5f46ec6022ce service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:33.249132 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef56d6-ca6b-481a-a960-5f46ec6022ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:33.249627 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef56d6-ca6b-481a-a960-5f46ec6022ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:33.253023 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a6ef56d6-ca6b-481a-a960-5f46ec6022ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:33.276563 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef56d6-ca6b-481a-a960-5f46ec6022ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:33.278054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ef56d6-ca6b-481a-a960-5f46ec6022ce service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:33.287971 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 871/1744] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:33.289488 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4823191-9b83-4c10-9840-7488811dcbd8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:33.289781 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4823191-9b83-4c10-9840-7488811dcbd8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:33.290071 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4823191-9b83-4c10-9840-7488811dcbd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:33.304514 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 874/1745] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jun 29 12:33:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:33.306528 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbaa5ee-55b1-46dc-aa4c-d024249951a5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:33.307096 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbaa5ee-55b1-46dc-aa4c-d024249951a5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:33.307580 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbaa5ee-55b1-46dc-aa4c-d024249951a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:33.335100 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d4823191-9b83-4c10-9840-7488811dcbd8 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:33.336330 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d4823191-9b83-4c10-9840-7488811dcbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:33.347470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8dbaa5ee-55b1-46dc-aa4c-d024249951a5 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:33.350289 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8dbaa5ee-55b1-46dc-aa4c-d024249951a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:33.420932 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 872/1746] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:33.421804 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4823191-9b83-4c10-9840-7488811dcbd8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:33.430600 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d4823191-9b83-4c10-9840-7488811dcbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:33.443713 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:33.451248 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4823191-9b83-4c10-9840-7488811dcbd8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:33.456623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 875/1747] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:33.459304 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbaa5ee-55b1-46dc-aa4c-d024249951a5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:33.474684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbaa5ee-55b1-46dc-aa4c-d024249951a5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:33.474805 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11045a7-e852-4002-8af4-746af63d2a3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:33.475411 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11045a7-e852-4002-8af4-746af63d2a3b service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:33.477168 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11045a7-e852-4002-8af4-746af63d2a3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:33.477467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11045a7-e852-4002-8af4-746af63d2a3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:33.479780 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d6080e-f401-4086-9bbb-aa3cb1871f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:33.480444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6080e-f401-4086-9bbb-aa3cb1871f46 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:33.480737 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6080e-f401-4086-9bbb-aa3cb1871f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:33.481018 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d6080e-f401-4086-9bbb-aa3cb1871f46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:33.482279 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d6080e-f401-4086-9bbb-aa3cb1871f46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:33:33.482639 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d6080e-f401-4086-9bbb-aa3cb1871f46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:33.484706 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d6080e-f401-4086-9bbb-aa3cb1871f46 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:33.485715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11045a7-e852-4002-8af4-746af63d2a3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:33.487219 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11045a7-e852-4002-8af4-746af63d2a3b service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:33.500680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 876/1748] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:33:33.502299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb395ac-63c6-4a05-9326-4177d63d91f0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:33.502815 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb395ac-63c6-4a05-9326-4177d63d91f0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:33.503104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb395ac-63c6-4a05-9326-4177d63d91f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:33.514476 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 873/1749] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:33.515459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11045a7-e852-4002-8af4-746af63d2a3b service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:33.536026 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11045a7-e852-4002-8af4-746af63d2a3b service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:33.537378 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7cb395ac-63c6-4a05-9326-4177d63d91f0 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:33.538738 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7cb395ac-63c6-4a05-9326-4177d63d91f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:33.551143 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 877/1750] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:33.552721 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb395ac-63c6-4a05-9326-4177d63d91f0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:33.554388 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7cb395ac-63c6-4a05-9326-4177d63d91f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:33.563338 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd0035-945c-4ede-9c81-12b90e6d8629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:33.564032 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd0035-945c-4ede-9c81-12b90e6d8629 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:33.564329 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd0035-945c-4ede-9c81-12b90e6d8629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:33.564618 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd0035-945c-4ede-9c81-12b90e6d8629 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:33.565484 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd0035-945c-4ede-9c81-12b90e6d8629 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:33.568334 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd0035-945c-4ede-9c81-12b90e6d8629 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:33.628704 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 874/1751] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:33 2022] POST /identity/v3/projects => generated 367 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:33.630017 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd0035-945c-4ede-9c81-12b90e6d8629 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:33.644680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb395ac-63c6-4a05-9326-4177d63d91f0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:33.694073 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:33.702040 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8db22-34c2-4d30-be8e-7e75cdd8c1e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:33.702759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8db22-34c2-4d30-be8e-7e75cdd8c1e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:33.703106 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8db22-34c2-4d30-be8e-7e75cdd8c1e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:33.703790 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8db22-34c2-4d30-be8e-7e75cdd8c1e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:33.706301 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bbb8db22-34c2-4d30-be8e-7e75cdd8c1e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:33.760201 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd0035-945c-4ede-9c81-12b90e6d8629 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:33.769324 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd32731-e92c-49a3-bd17-60d7696d8f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:33.770767 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd32731-e92c-49a3-bd17-60d7696d8f53 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:33.771218 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd32731-e92c-49a3-bd17-60d7696d8f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:33.771893 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd32731-e92c-49a3-bd17-60d7696d8f53 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:33.773205 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd32731-e92c-49a3-bd17-60d7696d8f53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:33:33.773902 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd32731-e92c-49a3-bd17-60d7696d8f53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:33.775759 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd32731-e92c-49a3-bd17-60d7696d8f53 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:33.783457 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8db22-34c2-4d30-be8e-7e75cdd8c1e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:33.785216 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8db22-34c2-4d30-be8e-7e75cdd8c1e7 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:33.802742 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 878/1752] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:33 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:33.813627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8db22-34c2-4d30-be8e-7e75cdd8c1e7 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:33.814235 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 875/1753] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:33:33.815154 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd32731-e92c-49a3-bd17-60d7696d8f53 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:33.837331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8db22-34c2-4d30-be8e-7e75cdd8c1e7 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:33.840449 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd32731-e92c-49a3-bd17-60d7696d8f53 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:33.849508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9e385-f632-46e7-a42f-4bc1ebe1d374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:33.849883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf92a55-bf61-4195-94f8-2e396c2aa0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:33.850656 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf92a55-bf61-4195-94f8-2e396c2aa0de admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:33.850871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf92a55-bf61-4195-94f8-2e396c2aa0de admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:33.851152 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf92a55-bf61-4195-94f8-2e396c2aa0de admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:33.851862 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf92a55-bf61-4195-94f8-2e396c2aa0de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:33.853432 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9e385-f632-46e7-a42f-4bc1ebe1d374 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:33.853721 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9e385-f632-46e7-a42f-4bc1ebe1d374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:33.854080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9e385-f632-46e7-a42f-4bc1ebe1d374 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:33.854721 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9e385-f632-46e7-a42f-4bc1ebe1d374 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:33.857075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf92a55-bf61-4195-94f8-2e396c2aa0de admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:33.860878 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9e385-f632-46e7-a42f-4bc1ebe1d374 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:33.939401 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 879/1754] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:33 2022] POST /identity/v3/users => generated 284 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:33.941473 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9e385-f632-46e7-a42f-4bc1ebe1d374 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:33.956317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fac9e385-f632-46e7-a42f-4bc1ebe1d374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:33.964030 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 876/1755] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:33 2022] POST /identity/v3/projects => generated 387 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:33.971211 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf92a55-bf61-4195-94f8-2e396c2aa0de admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:34.007421 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9e385-f632-46e7-a42f-4bc1ebe1d374 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:34.121548 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb615be-c374-4844-beb9-310e301f300a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:34.122981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb615be-c374-4844-beb9-310e301f300a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:34.123492 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb615be-c374-4844-beb9-310e301f300a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:34.123971 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb615be-c374-4844-beb9-310e301f300a admin admin] PATH_INFO: `/v3/projects/6d1e265893c14eafa75e67723215de7e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:34.125543 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb615be-c374-4844-beb9-310e301f300a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d1e265893c14eafa75e67723215de7e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:34.128453 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb615be-c374-4844-beb9-310e301f300a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:34.134737 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf92a55-bf61-4195-94f8-2e396c2aa0de admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:34.140356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4feabf-fc55-45eb-adfc-8dbaa6e658d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:34.141115 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4feabf-fc55-45eb-adfc-8dbaa6e658d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:34.141409 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4feabf-fc55-45eb-adfc-8dbaa6e658d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:34.141695 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4feabf-fc55-45eb-adfc-8dbaa6e658d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:34.142896 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4feabf-fc55-45eb-adfc-8dbaa6e658d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:34.156151 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4feabf-fc55-45eb-adfc-8dbaa6e658d6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:34.178113 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 877/1756] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:34.179254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4feabf-fc55-45eb-adfc-8dbaa6e658d6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:34.241425 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4feabf-fc55-45eb-adfc-8dbaa6e658d6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:34.294105 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-aeb615be-c374-4844-beb9-310e301f300a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d1e265893c14eafa75e67723215de7e', 'request_id': 'req-aeb615be-c374-4844-beb9-310e301f300a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:34.329032 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-aeb615be-c374-4844-beb9-310e301f300a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d1e265893c14eafa75e67723215de7e', 'request_id': 'req-aeb615be-c374-4844-beb9-310e301f300a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:34.344336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-aeb615be-c374-4844-beb9-310e301f300a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d1e265893c14eafa75e67723215de7e', 'request_id': 'req-aeb615be-c374-4844-beb9-310e301f300a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:34.365453 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:34.396009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-aeb615be-c374-4844-beb9-310e301f300a admin admin] The token cache is being invalidate because project 6d1e265893c14eafa75e67723215de7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:33:34.396920 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-aeb615be-c374-4844-beb9-310e301f300a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:34.398574 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-aeb615be-c374-4844-beb9-310e301f300a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:34.404913 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 880/1757] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:33:33 2022] DELETE /identity/v3/projects/6d1e265893c14eafa75e67723215de7e => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:34.404913 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb615be-c374-4844-beb9-310e301f300a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:34.407582 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-aeb615be-c374-4844-beb9-310e301f300a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:34.440553 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:34.447411 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34330191-db7a-4cfb-aac7-277de458a4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:34.450706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34330191-db7a-4cfb-aac7-277de458a4a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:34.451025 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34330191-db7a-4cfb-aac7-277de458a4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:34.451311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34330191-db7a-4cfb-aac7-277de458a4a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:34.458539 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34330191-db7a-4cfb-aac7-277de458a4a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:34.460540 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34330191-db7a-4cfb-aac7-277de458a4a2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:34.510228 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 878/1758] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:34 2022] POST /identity/v3/users => generated 294 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:34.511488 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34330191-db7a-4cfb-aac7-277de458a4a2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:34.514388 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-34330191-db7a-4cfb-aac7-277de458a4a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:34.544274 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34330191-db7a-4cfb-aac7-277de458a4a2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:34.552267 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac863dcb-5da2-4f18-8598-df17f2a0cca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:34.553582 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac863dcb-5da2-4f18-8598-df17f2a0cca3 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:34.554227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac863dcb-5da2-4f18-8598-df17f2a0cca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:34.555015 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac863dcb-5da2-4f18-8598-df17f2a0cca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:34.555877 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac863dcb-5da2-4f18-8598-df17f2a0cca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:34.560679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac863dcb-5da2-4f18-8598-df17f2a0cca3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:34.586369 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 879/1759] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:34.586369 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac863dcb-5da2-4f18-8598-df17f2a0cca3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:34.598671 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ac863dcb-5da2-4f18-8598-df17f2a0cca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:34.612392 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb615be-c374-4844-beb9-310e301f300a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:34.623207 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e58697-22a0-40de-825a-1baa6026a274 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98135cacb9d4f22846ad33500b9bd1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:34.624511 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e58697-22a0-40de-825a-1baa6026a274 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:34.625710 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e58697-22a0-40de-825a-1baa6026a274 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:34.626066 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e58697-22a0-40de-825a-1baa6026a274 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:34.631324 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-44e58697-22a0-40de-825a-1baa6026a274 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:34.662722 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e58697-22a0-40de-825a-1baa6026a274 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:34.664553 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e58697-22a0-40de-825a-1baa6026a274 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:34.686526 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 881/1760] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:34.686526 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e58697-22a0-40de-825a-1baa6026a274 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:34.691645 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-44e58697-22a0-40de-825a-1baa6026a274 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:34.718138 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e58697-22a0-40de-825a-1baa6026a274 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:34.725289 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65d479-f791-4abf-ad3a-17837af3d7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:34.726317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65d479-f791-4abf-ad3a-17837af3d7da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:34.726704 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65d479-f791-4abf-ad3a-17837af3d7da admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:34.727058 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65d479-f791-4abf-ad3a-17837af3d7da admin admin] PATH_INFO: `/v3/users/b3046ca9fef445489e588f239e9e4ad1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:34.728411 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65d479-f791-4abf-ad3a-17837af3d7da admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3046ca9fef445489e588f239e9e4ad1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:34.731144 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d65d479-f791-4abf-ad3a-17837af3d7da admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:34.773672 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac863dcb-5da2-4f18-8598-df17f2a0cca3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:34.780395 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307a2c6-c1ae-43da-8588-34e4da2a6389 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:34.781217 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307a2c6-c1ae-43da-8588-34e4da2a6389 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:34.782148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307a2c6-c1ae-43da-8588-34e4da2a6389 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:34.782574 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307a2c6-c1ae-43da-8588-34e4da2a6389 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:34.784790 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6307a2c6-c1ae-43da-8588-34e4da2a6389 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:34.810371 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6d65d479-f791-4abf-ad3a-17837af3d7da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3046ca9fef445489e588f239e9e4ad1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:34.811388 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6d65d479-f791-4abf-ad3a-17837af3d7da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3046ca9fef445489e588f239e9e4ad1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:34.859971 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6d65d479-f791-4abf-ad3a-17837af3d7da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3046ca9fef445489e588f239e9e4ad1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:34.861454 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6d65d479-f791-4abf-ad3a-17837af3d7da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3046ca9fef445489e588f239e9e4ad1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:34.875482 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6d65d479-f791-4abf-ad3a-17837af3d7da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3046ca9fef445489e588f239e9e4ad1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:34.895576 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 882/1761] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:33:34 2022] DELETE /identity/v3/users/b3046ca9fef445489e588f239e9e4ad1 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:34.895576 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65d479-f791-4abf-ad3a-17837af3d7da admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:34.895818 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6d65d479-f791-4abf-ad3a-17837af3d7da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:34.917991 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6307a2c6-c1ae-43da-8588-34e4da2a6389 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:34.919671 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6307a2c6-c1ae-43da-8588-34e4da2a6389 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:34.950568 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 880/1762] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:34.959627 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307a2c6-c1ae-43da-8588-34e4da2a6389 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:34.979354 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6307a2c6-c1ae-43da-8588-34e4da2a6389 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:34.980454 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65d479-f791-4abf-ad3a-17837af3d7da admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:34.996180 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab1d66-eff9-4690-b5ce-28a82309677d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:34.997337 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab1d66-eff9-4690-b5ce-28a82309677d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:34.998067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab1d66-eff9-4690-b5ce-28a82309677d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:34.999952 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab1d66-eff9-4690-b5ce-28a82309677d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:35.000742 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ab1d66-eff9-4690-b5ce-28a82309677d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:35.002384 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ab1d66-eff9-4690-b5ce-28a82309677d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:35.018359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307a2c6-c1ae-43da-8588-34e4da2a6389 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:35.067201 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 883/1763] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:35.068133 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab1d66-eff9-4690-b5ce-28a82309677d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:35.076683 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-49ab1d66-eff9-4690-b5ce-28a82309677d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:35.132408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:35.156643 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab1d66-eff9-4690-b5ce-28a82309677d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:35.191077 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f660a9c1-149c-46b7-86cf-b9e6043eda76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:35.193979 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660a9c1-149c-46b7-86cf-b9e6043eda76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:35.194269 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660a9c1-149c-46b7-86cf-b9e6043eda76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:35.196737 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:35.196911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f660a9c1-149c-46b7-86cf-b9e6043eda76 admin admin] PATH_INFO: `/v3/projects/00fbbc6be6ea4c77ab40a62d3f3f9ef3/users/d6fc7894de034f318f3db757dbcd2f8d/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:35.259954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627c2bd-ed40-4385-9612-021446121c01 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98135cacb9d4f22846ad33500b9bd1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:35.260656 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627c2bd-ed40-4385-9612-021446121c01 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:35.260936 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627c2bd-ed40-4385-9612-021446121c01 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:35.261221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627c2bd-ed40-4385-9612-021446121c01 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:35.262523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f660a9c1-149c-46b7-86cf-b9e6043eda76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00fbbc6be6ea4c77ab40a62d3f3f9ef3, user_id=d6fc7894de034f318f3db757dbcd2f8d, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:35.265416 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f660a9c1-149c-46b7-86cf-b9e6043eda76 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:35.282033 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5627c2bd-ed40-4385-9612-021446121c01 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:35.310610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 884/1764] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:35 2022] PUT /identity/v3/projects/00fbbc6be6ea4c77ab40a62d3f3f9ef3/users/d6fc7894de034f318f3db757dbcd2f8d/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:35.310610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f660a9c1-149c-46b7-86cf-b9e6043eda76 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:35.323904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627c2bd-ed40-4385-9612-021446121c01 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:35.325563 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627c2bd-ed40-4385-9612-021446121c01 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:35.384526 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 881/1765] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:35.386135 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1d71e-0c91-436b-ad6f-699044d56f24 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:35.386444 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1d71e-0c91-436b-ad6f-699044d56f24 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:35.386752 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1d71e-0c91-436b-ad6f-699044d56f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:35.535431 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-78f1d71e-0c91-436b-ad6f-699044d56f24 None None] MFA Rules not processed for user `d6fc7894de034f318f3db757dbcd2f8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:35.536270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f660a9c1-149c-46b7-86cf-b9e6043eda76 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:35.536686 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-78f1d71e-0c91-436b-ad6f-699044d56f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:35.556319 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313feca8-3673-4c22-b012-a4d5dbd24031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:35.557241 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313feca8-3673-4c22-b012-a4d5dbd24031 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:35.558222 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313feca8-3673-4c22-b012-a4d5dbd24031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:35.560249 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313feca8-3673-4c22-b012-a4d5dbd24031 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:35.561136 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313feca8-3673-4c22-b012-a4d5dbd24031 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:35.562924 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313feca8-3673-4c22-b012-a4d5dbd24031 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:35.625242 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 885/1766] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:35.626489 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313feca8-3673-4c22-b012-a4d5dbd24031 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:35.670013 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313feca8-3673-4c22-b012-a4d5dbd24031 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:35.688987 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c27727-4014-44c7-a7d2-bc10e1af343e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:35.698096 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c27727-4014-44c7-a7d2-bc10e1af343e admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:35.698567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c27727-4014-44c7-a7d2-bc10e1af343e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:35.698934 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c27727-4014-44c7-a7d2-bc10e1af343e admin admin] PATH_INFO: `/v3/projects/b3d7738358c0420c9e3a683e068cdbdc/users/6261822916f24e4bb847579ae2b7bbb8/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:35.757984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c27727-4014-44c7-a7d2-bc10e1af343e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3d7738358c0420c9e3a683e068cdbdc, user_id=6261822916f24e4bb847579ae2b7bbb8, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:35.758172 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 882/1767] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:35 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:35.762955 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c27727-4014-44c7-a7d2-bc10e1af343e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:35.764832 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1d71e-0c91-436b-ad6f-699044d56f24 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:35.809298 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1d71e-0c91-436b-ad6f-699044d56f24 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:35.828835 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 886/1768] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:35 2022] PUT /identity/v3/projects/b3d7738358c0420c9e3a683e068cdbdc/users/6261822916f24e4bb847579ae2b7bbb8/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:35.836126 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12addc57-9b8f-4dfd-aaad-03dbe86f465a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:35.836622 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12addc57-9b8f-4dfd-aaad-03dbe86f465a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:35.838258 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12addc57-9b8f-4dfd-aaad-03dbe86f465a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:36.010935 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8564510-55a2-415e-a82c-f3fbc5e79bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:36.011673 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8564510-55a2-415e-a82c-f3fbc5e79bf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:36.011961 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8564510-55a2-415e-a82c-f3fbc5e79bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:36.012250 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8564510-55a2-415e-a82c-f3fbc5e79bf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:36.012965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8564510-55a2-415e-a82c-f3fbc5e79bf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:36.015253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8564510-55a2-415e-a82c-f3fbc5e79bf8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:36.028117 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-12addc57-9b8f-4dfd-aaad-03dbe86f465a None None] MFA Rules not processed for user `6261822916f24e4bb847579ae2b7bbb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:36.032957 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-12addc57-9b8f-4dfd-aaad-03dbe86f465a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:36.093231 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 883/1769] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:35 2022] POST /identity/v3/projects => generated 369 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:36.098832 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7f4d9-7667-4d77-b5e2-8afdf1eb791e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:36.099051 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7f4d9-7667-4d77-b5e2-8afdf1eb791e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:36.099334 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf7f4d9-7667-4d77-b5e2-8afdf1eb791e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:36.159292 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5cf7f4d9-7667-4d77-b5e2-8afdf1eb791e None None] MFA Rules not processed for user `350a57ee307a4419974dcecf37e466e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:36.161167 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5cf7f4d9-7667-4d77-b5e2-8afdf1eb791e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:36.292312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 887/1770] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:36.300645 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12addc57-9b8f-4dfd-aaad-03dbe86f465a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:36.339542 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12addc57-9b8f-4dfd-aaad-03dbe86f465a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:36.354253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:36.385090 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 884/1771] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:36.386107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7f4d9-7667-4d77-b5e2-8afdf1eb791e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:36.557637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:36.612096 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7716ed4-7c70-4adf-95e3-ac67eb4ff587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:36.617458 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7716ed4-7c70-4adf-95e3-ac67eb4ff587 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:36.617828 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7716ed4-7c70-4adf-95e3-ac67eb4ff587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:36.618123 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7716ed4-7c70-4adf-95e3-ac67eb4ff587 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:36.618915 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7716ed4-7c70-4adf-95e3-ac67eb4ff587 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:36.626027 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7716ed4-7c70-4adf-95e3-ac67eb4ff587 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:36.640413 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf7f4d9-7667-4d77-b5e2-8afdf1eb791e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:36.651886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e647a-a414-4122-8049-fbc0db6ee4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:36.652579 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e647a-a414-4122-8049-fbc0db6ee4ff admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:36.652867 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e647a-a414-4122-8049-fbc0db6ee4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:36.653378 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e647a-a414-4122-8049-fbc0db6ee4ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:36.654202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e647a-a414-4122-8049-fbc0db6ee4ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:36.660413 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e647a-a414-4122-8049-fbc0db6ee4ff admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:36.676447 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 888/1772] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:36 2022] POST /identity/v3/users => generated 291 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:36.679464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7716ed4-7c70-4adf-95e3-ac67eb4ff587 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:36.683896 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d7716ed4-7c70-4adf-95e3-ac67eb4ff587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:36.715789 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7716ed4-7c70-4adf-95e3-ac67eb4ff587 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:36.750950 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcd13dd-9aed-4f81-b952-46e0b8911950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:36.755570 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcd13dd-9aed-4f81-b952-46e0b8911950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:36.757244 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 885/1773] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:36 2022] POST /identity/v3/projects => generated 387 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:36.758660 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcd13dd-9aed-4f81-b952-46e0b8911950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:36.759121 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcd13dd-9aed-4f81-b952-46e0b8911950 admin admin] PATH_INFO: `/v3/projects/e55414dc89c54939829e0ab66e8f37a7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:36.760098 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e647a-a414-4122-8049-fbc0db6ee4ff admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:36.762246 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcd13dd-9aed-4f81-b952-46e0b8911950 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e55414dc89c54939829e0ab66e8f37a7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:36.766109 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcd13dd-9aed-4f81-b952-46e0b8911950 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:36.990929 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cfcd13dd-9aed-4f81-b952-46e0b8911950 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e55414dc89c54939829e0ab66e8f37a7', 'request_id': 'req-cfcd13dd-9aed-4f81-b952-46e0b8911950'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:37.008076 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cfcd13dd-9aed-4f81-b952-46e0b8911950 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e55414dc89c54939829e0ab66e8f37a7', 'request_id': 'req-cfcd13dd-9aed-4f81-b952-46e0b8911950'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:37.034199 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cfcd13dd-9aed-4f81-b952-46e0b8911950 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e55414dc89c54939829e0ab66e8f37a7', 'request_id': 'req-cfcd13dd-9aed-4f81-b952-46e0b8911950'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:37.041046 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e647a-a414-4122-8049-fbc0db6ee4ff admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:37.081295 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cfcd13dd-9aed-4f81-b952-46e0b8911950 admin admin] The token cache is being invalidate because project e55414dc89c54939829e0ab66e8f37a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:33:37.081632 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cfcd13dd-9aed-4f81-b952-46e0b8911950 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:37.089245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cfcd13dd-9aed-4f81-b952-46e0b8911950 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:37.092124 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 889/1774] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:33:36 2022] DELETE /identity/v3/projects/e55414dc89c54939829e0ab66e8f37a7 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:37.099117 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcd13dd-9aed-4f81-b952-46e0b8911950 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:37.108391 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cfcd13dd-9aed-4f81-b952-46e0b8911950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:37.263888 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116d7ee6-c168-433c-be4a-c593e6eb4d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:37.264823 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d7ee6-c168-433c-be4a-c593e6eb4d3e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:37.270238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d7ee6-c168-433c-be4a-c593e6eb4d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:37.271014 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d7ee6-c168-433c-be4a-c593e6eb4d3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:37.272012 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d7ee6-c168-433c-be4a-c593e6eb4d3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:37.273994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116d7ee6-c168-433c-be4a-c593e6eb4d3e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:37.318836 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 886/1775] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 561 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:37.321537 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db568dbe-9faf-4c41-939b-4258e3234630 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:37.321537 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db568dbe-9faf-4c41-939b-4258e3234630 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:37.321537 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db568dbe-9faf-4c41-939b-4258e3234630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:37.480606 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcd13dd-9aed-4f81-b952-46e0b8911950 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:37.535189 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6ec3c-f197-4b74-a1f6-6792e4bcb2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:37.536498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6ec3c-f197-4b74-a1f6-6792e4bcb2ee admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:37.536758 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6ec3c-f197-4b74-a1f6-6792e4bcb2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:37.537200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6ec3c-f197-4b74-a1f6-6792e4bcb2ee admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:37.537930 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af6ec3c-f197-4b74-a1f6-6792e4bcb2ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:37.539943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af6ec3c-f197-4b74-a1f6-6792e4bcb2ee admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:37.640007 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 890/1776] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:37 2022] POST /identity/v3/users => generated 300 bytes in 541 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:37.640997 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6ec3c-f197-4b74-a1f6-6792e4bcb2ee admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:37.649446 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7af6ec3c-f197-4b74-a1f6-6792e4bcb2ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:37.688865 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6ec3c-f197-4b74-a1f6-6792e4bcb2ee admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:37.714259 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e3ba8-9363-4099-90fb-9e93d9efa5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:37.715235 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e3ba8-9363-4099-90fb-9e93d9efa5a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:37.715525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e3ba8-9363-4099-90fb-9e93d9efa5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:37.715908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e3ba8-9363-4099-90fb-9e93d9efa5a6 admin admin] PATH_INFO: `/v3/projects/2727758125054870aeff24637f71b1c1/users/230eec29b1464006bb5d43c7ade31948/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:37.771661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e3ba8-9363-4099-90fb-9e93d9efa5a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2727758125054870aeff24637f71b1c1, user_id=230eec29b1464006bb5d43c7ade31948, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:37.774658 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e3ba8-9363-4099-90fb-9e93d9efa5a6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:37.825735 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 891/1777] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:37 2022] PUT /identity/v3/projects/2727758125054870aeff24637f71b1c1/users/230eec29b1464006bb5d43c7ade31948/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:37.826816 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e3ba8-9363-4099-90fb-9e93d9efa5a6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:37.906346 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=85,keystone:INSERT=8,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:37.929856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-db568dbe-9faf-4c41-939b-4258e3234630 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:37.932368 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-db568dbe-9faf-4c41-939b-4258e3234630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:38.022112 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:38.085006 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e3ba8-9363-4099-90fb-9e93d9efa5a6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:38.113628 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845696a8-192b-4dfd-92e4-f2894612a747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:38.114384 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845696a8-192b-4dfd-92e4-f2894612a747 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:38.114824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845696a8-192b-4dfd-92e4-f2894612a747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:38.119181 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845696a8-192b-4dfd-92e4-f2894612a747 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:38.120147 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845696a8-192b-4dfd-92e4-f2894612a747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:38.131816 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845696a8-192b-4dfd-92e4-f2894612a747 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:38.187036 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 892/1778] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:38.188012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845696a8-192b-4dfd-92e4-f2894612a747 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:38.208471 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 887/1779] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 889 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:38.209281 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db568dbe-9faf-4c41-939b-4258e3234630 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:38.217413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845696a8-192b-4dfd-92e4-f2894612a747 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:38.228161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b425803a-c3b9-41ad-bcf3-140cb9b273c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:38.228860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425803a-c3b9-41ad-bcf3-140cb9b273c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:38.229151 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425803a-c3b9-41ad-bcf3-140cb9b273c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:38.230005 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b425803a-c3b9-41ad-bcf3-140cb9b273c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:38.230792 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b425803a-c3b9-41ad-bcf3-140cb9b273c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:38.232417 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b425803a-c3b9-41ad-bcf3-140cb9b273c5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:38.244909 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db568dbe-9faf-4c41-939b-4258e3234630 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:38.274812 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13935118-5afe-41a9-9eee-a32fc6f60702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:38.275992 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13935118-5afe-41a9-9eee-a32fc6f60702 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:38.276100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 893/1780] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:38.277490 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520596e2-5765-4d39-b8cf-e6f70fd753cb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:38.277784 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520596e2-5765-4d39-b8cf-e6f70fd753cb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:38.278075 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520596e2-5765-4d39-b8cf-e6f70fd753cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:38.278260 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13935118-5afe-41a9-9eee-a32fc6f60702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:38.278708 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13935118-5afe-41a9-9eee-a32fc6f60702 admin admin] PATH_INFO: `/v3/projects/da7ccabd53b04e33a1634ff74bcfa4eb/users/307bde8a4d56486ebeb1bb5275479b5b/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:38.325368 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13935118-5afe-41a9-9eee-a32fc6f60702 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da7ccabd53b04e33a1634ff74bcfa4eb, user_id=307bde8a4d56486ebeb1bb5275479b5b, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:38.331664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-520596e2-5765-4d39-b8cf-e6f70fd753cb None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:38.333090 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-520596e2-5765-4d39-b8cf-e6f70fd753cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:38.336893 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13935118-5afe-41a9-9eee-a32fc6f60702 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:38.361390 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 894/1781] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:38.361390 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520596e2-5765-4d39-b8cf-e6f70fd753cb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:38.375969 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 888/1782] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:38 2022] PUT /identity/v3/projects/da7ccabd53b04e33a1634ff74bcfa4eb/users/307bde8a4d56486ebeb1bb5275479b5b/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:38.377057 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13935118-5afe-41a9-9eee-a32fc6f60702 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:38.391892 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520596e2-5765-4d39-b8cf-e6f70fd753cb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:38.524195 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2558067-accc-484b-94e0-2d7f87e5d0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:38.525816 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2558067-accc-484b-94e0-2d7f87e5d0ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:38.526382 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2558067-accc-484b-94e0-2d7f87e5d0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:38.527101 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2558067-accc-484b-94e0-2d7f87e5d0ef admin admin] PATH_INFO: `/v3/projects/2727758125054870aeff24637f71b1c1/users/230eec29b1464006bb5d43c7ade31948/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:38.530019 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2558067-accc-484b-94e0-2d7f87e5d0ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=2727758125054870aeff24637f71b1c1, user_id=230eec29b1464006bb5d43c7ade31948, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:38.533111 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2558067-accc-484b-94e0-2d7f87e5d0ef admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:38.551566 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 895/1783] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:38 2022] PUT /identity/v3/projects/2727758125054870aeff24637f71b1c1/users/230eec29b1464006bb5d43c7ade31948/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:38.552750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2558067-accc-484b-94e0-2d7f87e5d0ef admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:38.566498 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13935118-5afe-41a9-9eee-a32fc6f60702 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:38.676672 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5529620-4259-40e8-935b-fbd6edc9af66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:38.676672 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5529620-4259-40e8-935b-fbd6edc9af66 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:38.676672 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5529620-4259-40e8-935b-fbd6edc9af66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:38.676672 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5529620-4259-40e8-935b-fbd6edc9af66 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:38.679185 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5529620-4259-40e8-935b-fbd6edc9af66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:33:38.679639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5529620-4259-40e8-935b-fbd6edc9af66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:38.681963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5529620-4259-40e8-935b-fbd6edc9af66 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:38.699240 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 889/1784] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:33:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:33:38.702680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51843de0-f482-4e7f-8cec-3806b6ab052f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:38.702976 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51843de0-f482-4e7f-8cec-3806b6ab052f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:38.703313 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51843de0-f482-4e7f-8cec-3806b6ab052f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:38.711189 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2558067-accc-484b-94e0-2d7f87e5d0ef admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:38.717815 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481d344-2142-43d4-bc4b-3d0c2a3761dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:38.718744 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481d344-2142-43d4-bc4b-3d0c2a3761dd admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:38.719165 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481d344-2142-43d4-bc4b-3d0c2a3761dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:38.719664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f481d344-2142-43d4-bc4b-3d0c2a3761dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:38.720963 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481d344-2142-43d4-bc4b-3d0c2a3761dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:38.724087 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f481d344-2142-43d4-bc4b-3d0c2a3761dd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:38.744814 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 896/1785] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:38 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:38.746016 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481d344-2142-43d4-bc4b-3d0c2a3761dd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:38.759051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f481d344-2142-43d4-bc4b-3d0c2a3761dd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:38.764693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206d9a7-30c7-4c89-bfa0-320abba90efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:38.765555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206d9a7-30c7-4c89-bfa0-320abba90efa admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:38.766712 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206d9a7-30c7-4c89-bfa0-320abba90efa admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:38.766989 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206d9a7-30c7-4c89-bfa0-320abba90efa admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:38.767971 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206d9a7-30c7-4c89-bfa0-320abba90efa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:38.771014 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206d9a7-30c7-4c89-bfa0-320abba90efa admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:38.772432 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-51843de0-f482-4e7f-8cec-3806b6ab052f None None] MFA Rules not processed for user `230eec29b1464006bb5d43c7ade31948`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:38.774974 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-51843de0-f482-4e7f-8cec-3806b6ab052f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:38.809215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 897/1786] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:38 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:38.810688 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206d9a7-30c7-4c89-bfa0-320abba90efa admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:38.826063 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:38.856144 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:38.950659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 890/1787] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:38 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:38.951800 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51843de0-f482-4e7f-8cec-3806b6ab052f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:38.975802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51843de0-f482-4e7f-8cec-3806b6ab052f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:39.013345 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206d9a7-30c7-4c89-bfa0-320abba90efa admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:39.023575 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140d2e8-d85a-4497-9165-0d3b7a5d15a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:39.025690 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140d2e8-d85a-4497-9165-0d3b7a5d15a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:39.026211 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140d2e8-d85a-4497-9165-0d3b7a5d15a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:39.026740 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140d2e8-d85a-4497-9165-0d3b7a5d15a3 admin admin] PATH_INFO: `/v3/projects/da7ccabd53b04e33a1634ff74bcfa4eb/users/307bde8a4d56486ebeb1bb5275479b5b/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:39.029127 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c140d2e8-d85a-4497-9165-0d3b7a5d15a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da7ccabd53b04e33a1634ff74bcfa4eb, user_id=307bde8a4d56486ebeb1bb5275479b5b, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:39.034319 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c140d2e8-d85a-4497-9165-0d3b7a5d15a3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:39.053946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 898/1788] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:38 2022] PUT /identity/v3/projects/da7ccabd53b04e33a1634ff74bcfa4eb/users/307bde8a4d56486ebeb1bb5275479b5b/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:39.055295 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140d2e8-d85a-4497-9165-0d3b7a5d15a3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:39.058329 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c140d2e8-d85a-4497-9165-0d3b7a5d15a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:39.103198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685e087-116e-4583-80a2-98bd6827b40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:39.103702 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685e087-116e-4583-80a2-98bd6827b40e admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:39.103993 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685e087-116e-4583-80a2-98bd6827b40e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:39.104284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685e087-116e-4583-80a2-98bd6827b40e admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:39.105734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c685e087-116e-4583-80a2-98bd6827b40e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:39.107731 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c685e087-116e-4583-80a2-98bd6827b40e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:39.168673 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 891/1789] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:38 2022] POST /identity/v3/users => generated 286 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:39.168673 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d19e6-59b8-4b55-8cc0-1842c9f59cdf None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:39.168673 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d19e6-59b8-4b55-8cc0-1842c9f59cdf None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:39.168673 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d19e6-59b8-4b55-8cc0-1842c9f59cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:39.227541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140d2e8-d85a-4497-9165-0d3b7a5d15a3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:39.243525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a2081-2978-4907-a950-89f039d60f0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:39.248464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a2081-2978-4907-a950-89f039d60f0c service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:39.250915 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a2081-2978-4907-a950-89f039d60f0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:39.252497 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a2081-2978-4907-a950-89f039d60f0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:39.323048 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b47d19e6-59b8-4b55-8cc0-1842c9f59cdf None None] MFA Rules not processed for user `307bde8a4d56486ebeb1bb5275479b5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:39.324299 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b47d19e6-59b8-4b55-8cc0-1842c9f59cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:39.400765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099a2081-2978-4907-a950-89f039d60f0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:39.409198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099a2081-2978-4907-a950-89f039d60f0c service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:39.475606 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 899/1790] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:39 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:39.476704 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a2081-2978-4907-a950-89f039d60f0c service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:39.650307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 892/1791] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 481 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:39.756110 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d19e6-59b8-4b55-8cc0-1842c9f59cdf None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:39.757632 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b47d19e6-59b8-4b55-8cc0-1842c9f59cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:39.788045 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d19e6-59b8-4b55-8cc0-1842c9f59cdf None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:39.793543 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707f8d6-ff17-4a7c-aa17-9dd8c3b6c2ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:39.799268 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707f8d6-ff17-4a7c-aa17-9dd8c3b6c2ae service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:39.799611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707f8d6-ff17-4a7c-aa17-9dd8c3b6c2ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:39.799902 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707f8d6-ff17-4a7c-aa17-9dd8c3b6c2ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:39.812994 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:39.843205 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6707f8d6-ff17-4a7c-aa17-9dd8c3b6c2ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:39.844698 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6707f8d6-ff17-4a7c-aa17-9dd8c3b6c2ae service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:39.877221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a2081-2978-4907-a950-89f039d60f0c service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:39.881345 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 893/1792] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:39.909807 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72124265-7189-4e72-b1d8-d16986eda9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:39.909979 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72124265-7189-4e72-b1d8-d16986eda9a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:39.912645 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72124265-7189-4e72-b1d8-d16986eda9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:39.912954 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72124265-7189-4e72-b1d8-d16986eda9a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:39.913726 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72124265-7189-4e72-b1d8-d16986eda9a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:39.915446 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72124265-7189-4e72-b1d8-d16986eda9a5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:39.975949 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 900/1793] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 500 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:39.979696 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707f8d6-ff17-4a7c-aa17-9dd8c3b6c2ae service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:39.997735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707f8d6-ff17-4a7c-aa17-9dd8c3b6c2ae service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:40.009294 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ebbf4-cfa0-4534-8817-16157241e49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:40.010240 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ebbf4-cfa0-4534-8817-16157241e49d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:40.010563 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ebbf4-cfa0-4534-8817-16157241e49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:40.010855 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ebbf4-cfa0-4534-8817-16157241e49d admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:40.011600 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ebbf4-cfa0-4534-8817-16157241e49d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:40.015072 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ebbf4-cfa0-4534-8817-16157241e49d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:40.043304 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 894/1794] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:40.047328 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72124265-7189-4e72-b1d8-d16986eda9a5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:40.063947 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72124265-7189-4e72-b1d8-d16986eda9a5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:40.087694 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc01f5-0b5a-4752-8ce4-91df4f2dd4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:40.088616 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc01f5-0b5a-4752-8ce4-91df4f2dd4c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:40.088907 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc01f5-0b5a-4752-8ce4-91df4f2dd4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:40.089189 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc01f5-0b5a-4752-8ce4-91df4f2dd4c3 admin admin] PATH_INFO: `/v3/projects/5fa7ab2c1350473998ab01772d1faa82/users/e474740f8d7e440ea9424e97adf0a5bc/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:40.139872 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc01f5-0b5a-4752-8ce4-91df4f2dd4c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fa7ab2c1350473998ab01772d1faa82, user_id=e474740f8d7e440ea9424e97adf0a5bc, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:40.142601 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc01f5-0b5a-4752-8ce4-91df4f2dd4c3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:40.176465 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 901/1795] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:40 2022] PUT /identity/v3/projects/5fa7ab2c1350473998ab01772d1faa82/users/e474740f8d7e440ea9424e97adf0a5bc/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:40.187369 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b9d98-278d-43d7-9396-0bdd2705cf81 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:40.187718 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b9d98-278d-43d7-9396-0bdd2705cf81 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:40.187948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b9d98-278d-43d7-9396-0bdd2705cf81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:40.296639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8e1b9d98-278d-43d7-9396-0bdd2705cf81 None None] MFA Rules not processed for user `e474740f8d7e440ea9424e97adf0a5bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:40.297890 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8e1b9d98-278d-43d7-9396-0bdd2705cf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:40.319930 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:40.350783 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d3c9b-58b2-40b9-8db4-ce2652e3bfcc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:40.351138 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d3c9b-58b2-40b9-8db4-ce2652e3bfcc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:40.351558 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d3c9b-58b2-40b9-8db4-ce2652e3bfcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:40.427941 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-097d3c9b-58b2-40b9-8db4-ce2652e3bfcc None None] MFA Rules not processed for user `8c10852a728844c6ae0a48e3772d95b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:40.432200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-097d3c9b-58b2-40b9-8db4-ce2652e3bfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:40.585483 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 895/1796] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:40.586360 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b9d98-278d-43d7-9396-0bdd2705cf81 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:40.652927 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 902/1797] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jun 29 12:33:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:40.654503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d3c9b-58b2-40b9-8db4-ce2652e3bfcc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:40.824979 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b9d98-278d-43d7-9396-0bdd2705cf81 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:40.837454 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfc01f-0b82-4145-93b0-021c3da72b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:40.838352 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfc01f-0b82-4145-93b0-021c3da72b9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:40.838667 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfc01f-0b82-4145-93b0-021c3da72b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:40.838947 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfc01f-0b82-4145-93b0-021c3da72b9e admin admin] PATH_INFO: `/v3/users/6261822916f24e4bb847579ae2b7bbb8` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:40.841169 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bfc01f-0b82-4145-93b0-021c3da72b9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6261822916f24e4bb847579ae2b7bbb8)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:40.844211 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bfc01f-0b82-4145-93b0-021c3da72b9e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:40.879771 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d3c9b-58b2-40b9-8db4-ce2652e3bfcc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:40.895994 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2722e8-517f-416d-88c6-060d09e9e8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:40.896860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2722e8-517f-416d-88c6-060d09e9e8ca admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:40.897217 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2722e8-517f-416d-88c6-060d09e9e8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:40.897572 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2722e8-517f-416d-88c6-060d09e9e8ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:40.898384 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2722e8-517f-416d-88c6-060d09e9e8ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:40.902676 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2722e8-517f-416d-88c6-060d09e9e8ca admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:40.950756 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d9bfc01f-0b82-4145-93b0-021c3da72b9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6261822916f24e4bb847579ae2b7bbb8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:40.970198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 903/1798] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:40 2022] POST /identity/v3/projects => generated 373 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:40.978935 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2722e8-517f-416d-88c6-060d09e9e8ca admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:40.979750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d9bfc01f-0b82-4145-93b0-021c3da72b9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6261822916f24e4bb847579ae2b7bbb8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:41.021289 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d9bfc01f-0b82-4145-93b0-021c3da72b9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6261822916f24e4bb847579ae2b7bbb8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:41.023211 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d9bfc01f-0b82-4145-93b0-021c3da72b9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6261822916f24e4bb847579ae2b7bbb8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:41.031061 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d9bfc01f-0b82-4145-93b0-021c3da72b9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6261822916f24e4bb847579ae2b7bbb8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:41.062799 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 896/1799] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:33:40 2022] DELETE /identity/v3/users/6261822916f24e4bb847579ae2b7bbb8 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:41.063914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfc01f-0b82-4145-93b0-021c3da72b9e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:41.067698 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d9bfc01f-0b82-4145-93b0-021c3da72b9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:41.109042 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:41.121905 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2722e8-517f-416d-88c6-060d09e9e8ca admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:41.136842 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:41.215577 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfc01f-0b82-4145-93b0-021c3da72b9e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:41.225482 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc538b2-a0fa-423e-b830-7bdbd27b5144 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209b0df45f76404da880a9ae0c637d67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:41.226229 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc538b2-a0fa-423e-b830-7bdbd27b5144 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:41.226693 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc538b2-a0fa-423e-b830-7bdbd27b5144 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:41.227074 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc538b2-a0fa-423e-b830-7bdbd27b5144 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:41.232002 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3bc538b2-a0fa-423e-b830-7bdbd27b5144 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:41.235417 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e72f58-07e3-4d44-a05c-a59a7354dde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:41.236064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e72f58-07e3-4d44-a05c-a59a7354dde1 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:41.236354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e72f58-07e3-4d44-a05c-a59a7354dde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:41.236775 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e72f58-07e3-4d44-a05c-a59a7354dde1 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:41.237540 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e72f58-07e3-4d44-a05c-a59a7354dde1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:41.241583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e72f58-07e3-4d44-a05c-a59a7354dde1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:41.288218 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 904/1800] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:40 2022] POST /identity/v3/users => generated 293 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:41.290147 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a2431-3299-4d57-9e9f-33b9c395ca47 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:41.290472 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a2431-3299-4d57-9e9f-33b9c395ca47 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:41.290748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a2431-3299-4d57-9e9f-33b9c395ca47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:41.343654 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-815a2431-3299-4d57-9e9f-33b9c395ca47 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:41.349624 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-815a2431-3299-4d57-9e9f-33b9c395ca47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:41.372893 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 905/1801] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:41.373767 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a2431-3299-4d57-9e9f-33b9c395ca47 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:41.376895 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-815a2431-3299-4d57-9e9f-33b9c395ca47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:41.388719 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc538b2-a0fa-423e-b830-7bdbd27b5144 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:41.391009 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc538b2-a0fa-423e-b830-7bdbd27b5144 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:41.411247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 897/1802] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 12:33:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 12:33:41.412336 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc538b2-a0fa-423e-b830-7bdbd27b5144 service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:41.414958 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a2431-3299-4d57-9e9f-33b9c395ca47 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:41.420942 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189160d-80df-47f5-8f80-29443e350263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:41.421800 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189160d-80df-47f5-8f80-29443e350263 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:41.422264 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189160d-80df-47f5-8f80-29443e350263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:41.422651 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189160d-80df-47f5-8f80-29443e350263 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:41.423463 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189160d-80df-47f5-8f80-29443e350263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:41.423867 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3bc538b2-a0fa-423e-b830-7bdbd27b5144 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:41.425918 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189160d-80df-47f5-8f80-29443e350263 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:41.439968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 906/1803] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:41.444009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107f28a-b719-45e9-807c-76508e9f5a79 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:41.448006 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107f28a-b719-45e9-807c-76508e9f5a79 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:41.448763 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107f28a-b719-45e9-807c-76508e9f5a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:41.487926 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4107f28a-b719-45e9-807c-76508e9f5a79 None None] MFA Rules not processed for user `a4a30b65a91a4fd89e8b59df670f60ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:41.493499 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4107f28a-b719-45e9-807c-76508e9f5a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:41.593326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc538b2-a0fa-423e-b830-7bdbd27b5144 service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:41.601748 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bedce5f-8590-4dde-8546-458cc628184c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:41.604422 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedce5f-8590-4dde-8546-458cc628184c service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:41.604757 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedce5f-8590-4dde-8546-458cc628184c service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:41.605048 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bedce5f-8590-4dde-8546-458cc628184c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:41.620611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bedce5f-8590-4dde-8546-458cc628184c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:41.623902 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bedce5f-8590-4dde-8546-458cc628184c service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:41.661794 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 898/1804] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:41.668232 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bedce5f-8590-4dde-8546-458cc628184c service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:41.695480 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 907/1805] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:41.698099 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bedce5f-8590-4dde-8546-458cc628184c service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:41.710129 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604c831-bc3d-4cde-90b5-339978d9d5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:41.714664 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604c831-bc3d-4cde-90b5-339978d9d5b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:41.715768 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604c831-bc3d-4cde-90b5-339978d9d5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:41.716733 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604c831-bc3d-4cde-90b5-339978d9d5b6 admin admin] PATH_INFO: `/v3/projects/10857d0ab8744d4c9c865c7f021ce9b2/users/15e4ed3829cb4f7f8bafd9f91c6adfc4/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:41.829717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604c831-bc3d-4cde-90b5-339978d9d5b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10857d0ab8744d4c9c865c7f021ce9b2, user_id=15e4ed3829cb4f7f8bafd9f91c6adfc4, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:41.843385 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604c831-bc3d-4cde-90b5-339978d9d5b6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:41.885024 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 899/1806] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:41 2022] PUT /identity/v3/projects/10857d0ab8744d4c9c865c7f021ce9b2/users/15e4ed3829cb4f7f8bafd9f91c6adfc4/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:41.888791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107f28a-b719-45e9-807c-76508e9f5a79 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:41.912332 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107f28a-b719-45e9-807c-76508e9f5a79 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:42.062160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6afa3b-0e77-4955-8b92-24fe982d7db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:42.062959 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6afa3b-0e77-4955-8b92-24fe982d7db1 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:42.063319 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6afa3b-0e77-4955-8b92-24fe982d7db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:42.063667 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6afa3b-0e77-4955-8b92-24fe982d7db1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:42.064486 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6afa3b-0e77-4955-8b92-24fe982d7db1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:42.071600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6afa3b-0e77-4955-8b92-24fe982d7db1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:42.091308 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 908/1807] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:42.106264 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604c831-bc3d-4cde-90b5-339978d9d5b6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:42.131247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604c831-bc3d-4cde-90b5-339978d9d5b6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:42.154328 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f784eb-9d63-46a5-b7cc-95f8539c8ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:42.155619 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f784eb-9d63-46a5-b7cc-95f8539c8ade admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:42.156104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f784eb-9d63-46a5-b7cc-95f8539c8ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:42.156594 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f784eb-9d63-46a5-b7cc-95f8539c8ade admin admin] PATH_INFO: `/v3/projects/10857d0ab8744d4c9c865c7f021ce9b2/users/15e4ed3829cb4f7f8bafd9f91c6adfc4/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:42.163607 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f784eb-9d63-46a5-b7cc-95f8539c8ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=10857d0ab8744d4c9c865c7f021ce9b2, user_id=15e4ed3829cb4f7f8bafd9f91c6adfc4, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:42.173504 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f784eb-9d63-46a5-b7cc-95f8539c8ade admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:42.226787 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 900/1808] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:42 2022] PUT /identity/v3/projects/10857d0ab8744d4c9c865c7f021ce9b2/users/15e4ed3829cb4f7f8bafd9f91c6adfc4/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:42.237128 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8ccc7-efbe-48ec-b98a-c9566db7f210 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:42.237504 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8ccc7-efbe-48ec-b98a-c9566db7f210 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:42.237860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8ccc7-efbe-48ec-b98a-c9566db7f210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:42.307021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:42.317208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-75a8ccc7-efbe-48ec-b98a-c9566db7f210 None None] MFA Rules not processed for user `15e4ed3829cb4f7f8bafd9f91c6adfc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:42.326646 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-75a8ccc7-efbe-48ec-b98a-c9566db7f210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:42.537549 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 909/1809] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:42 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:42.570746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f784eb-9d63-46a5-b7cc-95f8539c8ade admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:42.576688 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-73f784eb-9d63-46a5-b7cc-95f8539c8ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:42.622077 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8ccc7-efbe-48ec-b98a-c9566db7f210 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:42.624420 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-75a8ccc7-efbe-48ec-b98a-c9566db7f210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:42.769154 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:42.846608 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f784eb-9d63-46a5-b7cc-95f8539c8ade admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:42.849379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8ccc7-efbe-48ec-b98a-c9566db7f210 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:42.856747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd1580-42c2-4823-a672-4e69700976c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:42.858058 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd1580-42c2-4823-a672-4e69700976c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:42.858516 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd1580-42c2-4823-a672-4e69700976c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:42.859144 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd1580-42c2-4823-a672-4e69700976c7 admin admin] PATH_INFO: `/v3/projects/b3d7738358c0420c9e3a683e068cdbdc` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:42.859309 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83def77e-34a4-404a-93ad-cadc4c7f080f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:42.861078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83def77e-34a4-404a-93ad-cadc4c7f080f service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:42.861364 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83def77e-34a4-404a-93ad-cadc4c7f080f service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:42.861650 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83def77e-34a4-404a-93ad-cadc4c7f080f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:42.861808 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd1580-42c2-4823-a672-4e69700976c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3d7738358c0420c9e3a683e068cdbdc)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:42.872492 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd1580-42c2-4823-a672-4e69700976c7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:42.887819 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83def77e-34a4-404a-93ad-cadc4c7f080f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:42.891760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83def77e-34a4-404a-93ad-cadc4c7f080f service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:42.927717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 901/1810] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:42 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:43.018512 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-02fd1580-42c2-4823-a672-4e69700976c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3d7738358c0420c9e3a683e068cdbdc', 'request_id': 'req-02fd1580-42c2-4823-a672-4e69700976c7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:43.034555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-02fd1580-42c2-4823-a672-4e69700976c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3d7738358c0420c9e3a683e068cdbdc', 'request_id': 'req-02fd1580-42c2-4823-a672-4e69700976c7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:43.043528 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-02fd1580-42c2-4823-a672-4e69700976c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3d7738358c0420c9e3a683e068cdbdc', 'request_id': 'req-02fd1580-42c2-4823-a672-4e69700976c7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:43.060894 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-02fd1580-42c2-4823-a672-4e69700976c7 admin admin] The token cache is being invalidate because project b3d7738358c0420c9e3a683e068cdbdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:33:43.061348 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-02fd1580-42c2-4823-a672-4e69700976c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:43.062446 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-02fd1580-42c2-4823-a672-4e69700976c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:43.064372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 910/1811] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:33:42 2022] DELETE /identity/v3/projects/b3d7738358c0420c9e3a683e068cdbdc => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:43.068656 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83def77e-34a4-404a-93ad-cadc4c7f080f service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:43.083436 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83def77e-34a4-404a-93ad-cadc4c7f080f service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:43.284571 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdec4e5-3737-4911-8e41-1b556a9221c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:43.290476 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdec4e5-3737-4911-8e41-1b556a9221c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:43.293027 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdec4e5-3737-4911-8e41-1b556a9221c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:43.298288 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdec4e5-3737-4911-8e41-1b556a9221c7 admin admin] PATH_INFO: `/v3/users/307bde8a4d56486ebeb1bb5275479b5b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:43.302220 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdec4e5-3737-4911-8e41-1b556a9221c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=307bde8a4d56486ebeb1bb5275479b5b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:43.314763 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdec4e5-3737-4911-8e41-1b556a9221c7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:43.458944 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd1580-42c2-4823-a672-4e69700976c7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:43.460709 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-02fd1580-42c2-4823-a672-4e69700976c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:43.505131 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd1580-42c2-4823-a672-4e69700976c7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:43.513374 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3bdec4e5-3737-4911-8e41-1b556a9221c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '307bde8a4d56486ebeb1bb5275479b5b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:43.525678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33669a5e-a510-4a6b-9eeb-a5cd59493f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:43.526933 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33669a5e-a510-4a6b-9eeb-a5cd59493f75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:43.528465 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33669a5e-a510-4a6b-9eeb-a5cd59493f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:43.529382 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33669a5e-a510-4a6b-9eeb-a5cd59493f75 admin admin] PATH_INFO: `/v3/users/d6fc7894de034f318f3db757dbcd2f8d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:43.533347 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33669a5e-a510-4a6b-9eeb-a5cd59493f75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6fc7894de034f318f3db757dbcd2f8d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:43.546165 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3bdec4e5-3737-4911-8e41-1b556a9221c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '307bde8a4d56486ebeb1bb5275479b5b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:43.546287 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33669a5e-a510-4a6b-9eeb-a5cd59493f75 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:43.605828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3bdec4e5-3737-4911-8e41-1b556a9221c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '307bde8a4d56486ebeb1bb5275479b5b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:43.607076 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3bdec4e5-3737-4911-8e41-1b556a9221c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '307bde8a4d56486ebeb1bb5275479b5b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:43.608037 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3bdec4e5-3737-4911-8e41-1b556a9221c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '307bde8a4d56486ebeb1bb5275479b5b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:43.641844 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 902/1812] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:33:43 2022] DELETE /identity/v3/users/307bde8a4d56486ebeb1bb5275479b5b => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:43.694743 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:43.790301 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-33669a5e-a510-4a6b-9eeb-a5cd59493f75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6fc7894de034f318f3db757dbcd2f8d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:43.799434 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-33669a5e-a510-4a6b-9eeb-a5cd59493f75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6fc7894de034f318f3db757dbcd2f8d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:43.875218 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-33669a5e-a510-4a6b-9eeb-a5cd59493f75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6fc7894de034f318f3db757dbcd2f8d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:43.877256 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-33669a5e-a510-4a6b-9eeb-a5cd59493f75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6fc7894de034f318f3db757dbcd2f8d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:43.915744 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-33669a5e-a510-4a6b-9eeb-a5cd59493f75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6fc7894de034f318f3db757dbcd2f8d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:43.986312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 911/1813] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:33:43 2022] DELETE /identity/v3/users/d6fc7894de034f318f3db757dbcd2f8d => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:43.992102 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd853c3-89fe-4501-88b5-8b914f65ca7e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:43.992673 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd853c3-89fe-4501-88b5-8b914f65ca7e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:43.992876 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd853c3-89fe-4501-88b5-8b914f65ca7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:44.063197 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5fd853c3-89fe-4501-88b5-8b914f65ca7e None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:44.065836 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5fd853c3-89fe-4501-88b5-8b914f65ca7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:44.241072 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:44.257801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 903/1814] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:44.299450 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33669a5e-a510-4a6b-9eeb-a5cd59493f75 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:44.301076 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-33669a5e-a510-4a6b-9eeb-a5cd59493f75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:44.333943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33669a5e-a510-4a6b-9eeb-a5cd59493f75 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:44.339201 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd853c3-89fe-4501-88b5-8b914f65ca7e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:44.340952 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5fd853c3-89fe-4501-88b5-8b914f65ca7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:44.369596 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e335a515-b3d9-4974-9f86-453b7a8d6899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:44.374657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335a515-b3d9-4974-9f86-453b7a8d6899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:44.376476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335a515-b3d9-4974-9f86-453b7a8d6899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:44.376796 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e335a515-b3d9-4974-9f86-453b7a8d6899 admin admin] PATH_INFO: `/v3/users/e474740f8d7e440ea9424e97adf0a5bc` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:44.385903 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e335a515-b3d9-4974-9f86-453b7a8d6899 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e474740f8d7e440ea9424e97adf0a5bc)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:44.392243 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e335a515-b3d9-4974-9f86-453b7a8d6899 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:44.552713 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd853c3-89fe-4501-88b5-8b914f65ca7e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:44.561695 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4e1a95-63b9-4915-bebe-aafac1177d1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:44.563461 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e1a95-63b9-4915-bebe-aafac1177d1b service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:44.563461 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e1a95-63b9-4915-bebe-aafac1177d1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:44.563461 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e1a95-63b9-4915-bebe-aafac1177d1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:44.573864 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4e1a95-63b9-4915-bebe-aafac1177d1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:44.575367 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4e1a95-63b9-4915-bebe-aafac1177d1b service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:44.588509 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 904/1815] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 12:33:44.589867 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb6bba-7e04-459c-92e3-c0e5983e1d95 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:44.590259 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb6bba-7e04-459c-92e3-c0e5983e1d95 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:44.590520 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb6bba-7e04-459c-92e3-c0e5983e1d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:44.627772 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-fbfb6bba-7e04-459c-92e3-c0e5983e1d95 None None] MFA Rules not processed for user `ce922cdbf2d34f10b6e49631f0c6d770`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:44.629123 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fbfb6bba-7e04-459c-92e3-c0e5983e1d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:44.656197 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 905/1816] 10.222.0.23 () {58 vars in 1015 bytes} [Wed Jun 29 12:33:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:44.666750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e335a515-b3d9-4974-9f86-453b7a8d6899 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e474740f8d7e440ea9424e97adf0a5bc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:44.667639 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e335a515-b3d9-4974-9f86-453b7a8d6899 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e474740f8d7e440ea9424e97adf0a5bc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:44.694590 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e335a515-b3d9-4974-9f86-453b7a8d6899 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e474740f8d7e440ea9424e97adf0a5bc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:44.695486 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e335a515-b3d9-4974-9f86-453b7a8d6899 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e474740f8d7e440ea9424e97adf0a5bc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:44.712770 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e335a515-b3d9-4974-9f86-453b7a8d6899 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e474740f8d7e440ea9424e97adf0a5bc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:44.748878 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 912/1817] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:33:44 2022] DELETE /identity/v3/users/e474740f8d7e440ea9424e97adf0a5bc => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:44.759076 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e9ea9-9cdc-4b60-9711-7afd2f8137b3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:44.759591 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e9ea9-9cdc-4b60-9711-7afd2f8137b3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:44.759996 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e9ea9-9cdc-4b60-9711-7afd2f8137b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:44.808083 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2c6e9ea9-9cdc-4b60-9711-7afd2f8137b3 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:44.821430 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2c6e9ea9-9cdc-4b60-9711-7afd2f8137b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:44.872264 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e335a515-b3d9-4974-9f86-453b7a8d6899 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:44.877145 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e335a515-b3d9-4974-9f86-453b7a8d6899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:45.147685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 906/1818] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:45.172609 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e335a515-b3d9-4974-9f86-453b7a8d6899 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:45.183059 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd30ffb-bc35-4066-91e2-ed482a8a04bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:45.183745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd30ffb-bc35-4066-91e2-ed482a8a04bf service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:45.184263 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd30ffb-bc35-4066-91e2-ed482a8a04bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:45.187503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd30ffb-bc35-4066-91e2-ed482a8a04bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:45.203061 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e9ea9-9cdc-4b60-9711-7afd2f8137b3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:45.204833 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2c6e9ea9-9cdc-4b60-9711-7afd2f8137b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:45.209915 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7fd30ffb-bc35-4066-91e2-ed482a8a04bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:45.261185 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e9ea9-9cdc-4b60-9711-7afd2f8137b3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:45.263511 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd30ffb-bc35-4066-91e2-ed482a8a04bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:45.267366 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd30ffb-bc35-4066-91e2-ed482a8a04bf service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:45.278514 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c6a98-1fd2-4db6-86d0-ae1e10ec3db7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:45.278514 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c6a98-1fd2-4db6-86d0-ae1e10ec3db7 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:45.278514 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c6a98-1fd2-4db6-86d0-ae1e10ec3db7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:45.284158 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2c6a98-1fd2-4db6-86d0-ae1e10ec3db7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:45.322959 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 913/1819] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 12:33:45.337290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c6a98-1fd2-4db6-86d0-ae1e10ec3db7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:45.346330 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2c6a98-1fd2-4db6-86d0-ae1e10ec3db7 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:45.407587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 907/1820] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:46.798133 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd30ffb-bc35-4066-91e2-ed482a8a04bf service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:46.801300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7fd30ffb-bc35-4066-91e2-ed482a8a04bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:46.818660 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd30ffb-bc35-4066-91e2-ed482a8a04bf service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:46.824182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aacdf8-85aa-4eef-b83e-0b1b55e286c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:46.825091 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aacdf8-85aa-4eef-b83e-0b1b55e286c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:46.825387 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aacdf8-85aa-4eef-b83e-0b1b55e286c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:46.825670 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aacdf8-85aa-4eef-b83e-0b1b55e286c2 admin admin] PATH_INFO: `/v3/projects/5fa7ab2c1350473998ab01772d1faa82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:46.826928 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aacdf8-85aa-4eef-b83e-0b1b55e286c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fa7ab2c1350473998ab01772d1faa82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:46.829161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aacdf8-85aa-4eef-b83e-0b1b55e286c2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:46.841568 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:46.900061 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e2aacdf8-85aa-4eef-b83e-0b1b55e286c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fa7ab2c1350473998ab01772d1faa82', 'request_id': 'req-e2aacdf8-85aa-4eef-b83e-0b1b55e286c2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:46.912934 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e2aacdf8-85aa-4eef-b83e-0b1b55e286c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fa7ab2c1350473998ab01772d1faa82', 'request_id': 'req-e2aacdf8-85aa-4eef-b83e-0b1b55e286c2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:46.923308 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e2aacdf8-85aa-4eef-b83e-0b1b55e286c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fa7ab2c1350473998ab01772d1faa82', 'request_id': 'req-e2aacdf8-85aa-4eef-b83e-0b1b55e286c2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:46.955137 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e2aacdf8-85aa-4eef-b83e-0b1b55e286c2 admin admin] The token cache is being invalidate because project 5fa7ab2c1350473998ab01772d1faa82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:33:46.955665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e2aacdf8-85aa-4eef-b83e-0b1b55e286c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:46.956517 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e2aacdf8-85aa-4eef-b83e-0b1b55e286c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:46.959055 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 914/1821] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:33:46 2022] DELETE /identity/v3/projects/5fa7ab2c1350473998ab01772d1faa82 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:46.963374 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c6a98-1fd2-4db6-86d0-ae1e10ec3db7 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:46.974131 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2c6a98-1fd2-4db6-86d0-ae1e10ec3db7 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:47.003297 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aacdf8-85aa-4eef-b83e-0b1b55e286c2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:47.005619 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e2aacdf8-85aa-4eef-b83e-0b1b55e286c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:47.059361 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b43150-256f-44e3-9f02-d59b24f6dfa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:47.061437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b43150-256f-44e3-9f02-d59b24f6dfa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:47.061835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b43150-256f-44e3-9f02-d59b24f6dfa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:47.062223 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b43150-256f-44e3-9f02-d59b24f6dfa2 admin admin] PATH_INFO: `/v3/users/15e4ed3829cb4f7f8bafd9f91c6adfc4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:47.063991 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b43150-256f-44e3-9f02-d59b24f6dfa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15e4ed3829cb4f7f8bafd9f91c6adfc4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:47.067750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b43150-256f-44e3-9f02-d59b24f6dfa2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:47.103763 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aacdf8-85aa-4eef-b83e-0b1b55e286c2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:47.110698 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:47.110862 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528bf1e-586a-41b4-825d-b12c88939e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:47.111752 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528bf1e-586a-41b4-825d-b12c88939e57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:47.112044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528bf1e-586a-41b4-825d-b12c88939e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:47.112458 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528bf1e-586a-41b4-825d-b12c88939e57 admin admin] PATH_INFO: `/v3/projects/da7ccabd53b04e33a1634ff74bcfa4eb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:47.113937 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528bf1e-586a-41b4-825d-b12c88939e57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da7ccabd53b04e33a1634ff74bcfa4eb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:47.116293 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528bf1e-586a-41b4-825d-b12c88939e57 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:47.141746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-79b43150-256f-44e3-9f02-d59b24f6dfa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15e4ed3829cb4f7f8bafd9f91c6adfc4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:47.152845 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-79b43150-256f-44e3-9f02-d59b24f6dfa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15e4ed3829cb4f7f8bafd9f91c6adfc4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:47.169956 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-79b43150-256f-44e3-9f02-d59b24f6dfa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15e4ed3829cb4f7f8bafd9f91c6adfc4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:47.171131 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-79b43150-256f-44e3-9f02-d59b24f6dfa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15e4ed3829cb4f7f8bafd9f91c6adfc4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:47.172046 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-79b43150-256f-44e3-9f02-d59b24f6dfa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15e4ed3829cb4f7f8bafd9f91c6adfc4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:47.189641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 908/1822] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:33:46 2022] DELETE /identity/v3/users/15e4ed3829cb4f7f8bafd9f91c6adfc4 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:33:47.195431 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a528bf1e-586a-41b4-825d-b12c88939e57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da7ccabd53b04e33a1634ff74bcfa4eb', 'request_id': 'req-a528bf1e-586a-41b4-825d-b12c88939e57'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:47.204930 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a528bf1e-586a-41b4-825d-b12c88939e57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da7ccabd53b04e33a1634ff74bcfa4eb', 'request_id': 'req-a528bf1e-586a-41b4-825d-b12c88939e57'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:47.213341 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a528bf1e-586a-41b4-825d-b12c88939e57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da7ccabd53b04e33a1634ff74bcfa4eb', 'request_id': 'req-a528bf1e-586a-41b4-825d-b12c88939e57'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:47.237203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a528bf1e-586a-41b4-825d-b12c88939e57 admin admin] The token cache is being invalidate because project da7ccabd53b04e33a1634ff74bcfa4eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:33:47.240237 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a528bf1e-586a-41b4-825d-b12c88939e57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:47.242069 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a528bf1e-586a-41b4-825d-b12c88939e57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:47.246508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 915/1823] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:33:47 2022] DELETE /identity/v3/projects/da7ccabd53b04e33a1634ff74bcfa4eb => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:47.251762 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c9e18-6239-47bd-a337-0a43e15725d2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:47.252136 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c9e18-6239-47bd-a337-0a43e15725d2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:47.252493 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91c9e18-6239-47bd-a337-0a43e15725d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:47.276507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a91c9e18-6239-47bd-a337-0a43e15725d2 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:47.277910 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a91c9e18-6239-47bd-a337-0a43e15725d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:47.388333 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26cb9a-4262-463e-8f45-edb10338eda6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:47.388629 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26cb9a-4262-463e-8f45-edb10338eda6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:47.388906 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26cb9a-4262-463e-8f45-edb10338eda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:47.391764 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 909/1824] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:47.412103 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8f26cb9a-4262-463e-8f45-edb10338eda6 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:47.413479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8f26cb9a-4262-463e-8f45-edb10338eda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:47.423620 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 916/1825] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:47.428782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c9e18-6239-47bd-a337-0a43e15725d2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:47.439301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91c9e18-6239-47bd-a337-0a43e15725d2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:47.445629 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb66247-4fc9-4ff9-ac5f-97f2dad4a0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:47.447393 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb66247-4fc9-4ff9-ac5f-97f2dad4a0b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:47.448089 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb66247-4fc9-4ff9-ac5f-97f2dad4a0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:47.448516 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb66247-4fc9-4ff9-ac5f-97f2dad4a0b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:47.451873 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb66247-4fc9-4ff9-ac5f-97f2dad4a0b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:33:47.451873 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb66247-4fc9-4ff9-ac5f-97f2dad4a0b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:47.451873 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb66247-4fc9-4ff9-ac5f-97f2dad4a0b5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:47.473558 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26cb9a-4262-463e-8f45-edb10338eda6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:47.480405 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 910/1826] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:33:47.488514 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26cb9a-4262-463e-8f45-edb10338eda6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:47.497872 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c82f14-26a6-4f42-bbd8-e769bb3c153e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:47.499144 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c82f14-26a6-4f42-bbd8-e769bb3c153e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:47.499639 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c82f14-26a6-4f42-bbd8-e769bb3c153e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:47.500606 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c82f14-26a6-4f42-bbd8-e769bb3c153e admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:47.501328 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c82f14-26a6-4f42-bbd8-e769bb3c153e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:47.506835 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c82f14-26a6-4f42-bbd8-e769bb3c153e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:47.528757 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb66247-4fc9-4ff9-ac5f-97f2dad4a0b5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:47.531148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fdb66247-4fc9-4ff9-ac5f-97f2dad4a0b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:47.553030 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb66247-4fc9-4ff9-ac5f-97f2dad4a0b5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:47.569664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:47.574807 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4f40d-e24e-481c-aa1c-96f3e0e56b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:47.582924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4f40d-e24e-481c-aa1c-96f3e0e56b36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:47.583223 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4f40d-e24e-481c-aa1c-96f3e0e56b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:47.585330 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 917/1827] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:47 2022] POST /identity/v3/projects => generated 365 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:47.589784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4f40d-e24e-481c-aa1c-96f3e0e56b36 admin admin] PATH_INFO: `/v3/projects/00fbbc6be6ea4c77ab40a62d3f3f9ef3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:47.590037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c82f14-26a6-4f42-bbd8-e769bb3c153e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:47.594892 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4f40d-e24e-481c-aa1c-96f3e0e56b36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00fbbc6be6ea4c77ab40a62d3f3f9ef3)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:47.601354 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4f40d-e24e-481c-aa1c-96f3e0e56b36 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:47.699756 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c82f14-26a6-4f42-bbd8-e769bb3c153e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:47.731702 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8da4f40d-e24e-481c-aa1c-96f3e0e56b36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00fbbc6be6ea4c77ab40a62d3f3f9ef3', 'request_id': 'req-8da4f40d-e24e-481c-aa1c-96f3e0e56b36'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:47.740544 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8da4f40d-e24e-481c-aa1c-96f3e0e56b36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00fbbc6be6ea4c77ab40a62d3f3f9ef3', 'request_id': 'req-8da4f40d-e24e-481c-aa1c-96f3e0e56b36'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:47.749063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8da4f40d-e24e-481c-aa1c-96f3e0e56b36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00fbbc6be6ea4c77ab40a62d3f3f9ef3', 'request_id': 'req-8da4f40d-e24e-481c-aa1c-96f3e0e56b36'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:47.766334 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8da4f40d-e24e-481c-aa1c-96f3e0e56b36 admin admin] The token cache is being invalidate because project 00fbbc6be6ea4c77ab40a62d3f3f9ef3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:33:47.766801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8da4f40d-e24e-481c-aa1c-96f3e0e56b36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:47.767613 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8da4f40d-e24e-481c-aa1c-96f3e0e56b36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:47.773515 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 911/1828] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:33:47 2022] DELETE /identity/v3/projects/00fbbc6be6ea4c77ab40a62d3f3f9ef3 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:33:47.773515 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4f40d-e24e-481c-aa1c-96f3e0e56b36 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:47.773776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8da4f40d-e24e-481c-aa1c-96f3e0e56b36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:47.805425 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:47.921915 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4f40d-e24e-481c-aa1c-96f3e0e56b36 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:47.939324 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136c66a-ac45-4156-b177-53723db07db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:47.940303 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136c66a-ac45-4156-b177-53723db07db7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:47.940594 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136c66a-ac45-4156-b177-53723db07db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:47.941928 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136c66a-ac45-4156-b177-53723db07db7 admin admin] PATH_INFO: `/v3/projects/10857d0ab8744d4c9c865c7f021ce9b2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:47.944395 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9136c66a-ac45-4156-b177-53723db07db7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10857d0ab8744d4c9c865c7f021ce9b2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:47.947635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9136c66a-ac45-4156-b177-53723db07db7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:48.014364 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13da0af-f334-447d-85fc-15684a2bb7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:48.015768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13da0af-f334-447d-85fc-15684a2bb7a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:48.016054 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13da0af-f334-447d-85fc-15684a2bb7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:48.017612 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13da0af-f334-447d-85fc-15684a2bb7a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:48.018828 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13da0af-f334-447d-85fc-15684a2bb7a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:48.027265 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13da0af-f334-447d-85fc-15684a2bb7a9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:48.050443 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9136c66a-ac45-4156-b177-53723db07db7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10857d0ab8744d4c9c865c7f021ce9b2', 'request_id': 'req-9136c66a-ac45-4156-b177-53723db07db7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:48.075119 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 918/1829] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:47 2022] POST /identity/v3/users => generated 283 bytes in 486 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:48.076029 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13da0af-f334-447d-85fc-15684a2bb7a9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:48.078677 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9136c66a-ac45-4156-b177-53723db07db7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10857d0ab8744d4c9c865c7f021ce9b2', 'request_id': 'req-9136c66a-ac45-4156-b177-53723db07db7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:48.080272 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c13da0af-f334-447d-85fc-15684a2bb7a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:48.090233 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9136c66a-ac45-4156-b177-53723db07db7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10857d0ab8744d4c9c865c7f021ce9b2', 'request_id': 'req-9136c66a-ac45-4156-b177-53723db07db7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:48.093883 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13da0af-f334-447d-85fc-15684a2bb7a9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:48.131589 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9136c66a-ac45-4156-b177-53723db07db7 admin admin] The token cache is being invalidate because project 10857d0ab8744d4c9c865c7f021ce9b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:33:48.132264 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9136c66a-ac45-4156-b177-53723db07db7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:48.133916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9136c66a-ac45-4156-b177-53723db07db7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:48.140526 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 912/1830] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:33:47 2022] DELETE /identity/v3/projects/10857d0ab8744d4c9c865c7f021ce9b2 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:48.140526 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136c66a-ac45-4156-b177-53723db07db7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:48.141751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9136c66a-ac45-4156-b177-53723db07db7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:48.186856 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81064d36-ae22-4cc2-af94-87e1de643923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:48.187927 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81064d36-ae22-4cc2-af94-87e1de643923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:48.188222 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81064d36-ae22-4cc2-af94-87e1de643923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:48.188509 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81064d36-ae22-4cc2-af94-87e1de643923 admin admin] PATH_INFO: `/v3/users/230eec29b1464006bb5d43c7ade31948` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:48.189831 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81064d36-ae22-4cc2-af94-87e1de643923 admin admin] RBAC: Authorizing `identity:delete_user(user_id=230eec29b1464006bb5d43c7ade31948)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:48.191839 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81064d36-ae22-4cc2-af94-87e1de643923 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:48.264932 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:48.278873 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-81064d36-ae22-4cc2-af94-87e1de643923 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '230eec29b1464006bb5d43c7ade31948'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:48.279859 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-81064d36-ae22-4cc2-af94-87e1de643923 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '230eec29b1464006bb5d43c7ade31948'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:48.287573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136c66a-ac45-4156-b177-53723db07db7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:48.302849 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-81064d36-ae22-4cc2-af94-87e1de643923 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '230eec29b1464006bb5d43c7ade31948'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:48.303855 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-81064d36-ae22-4cc2-af94-87e1de643923 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '230eec29b1464006bb5d43c7ade31948'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:48.308827 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bef107-134d-42dc-9709-173e51c95848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:48.309846 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bef107-134d-42dc-9709-173e51c95848 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:48.310243 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bef107-134d-42dc-9709-173e51c95848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:48.310973 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bef107-134d-42dc-9709-173e51c95848 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:48.312346 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bef107-134d-42dc-9709-173e51c95848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:48.315327 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-81064d36-ae22-4cc2-af94-87e1de643923 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '230eec29b1464006bb5d43c7ade31948'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:48.320628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bef107-134d-42dc-9709-173e51c95848 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:48.334050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 913/1831] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:48.335566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9a1e6-b9d6-4471-9b21-dd94f4fb84f6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:48.335852 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9a1e6-b9d6-4471-9b21-dd94f4fb84f6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:48.336130 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9a1e6-b9d6-4471-9b21-dd94f4fb84f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:48.341437 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 919/1832] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:33:48 2022] DELETE /identity/v3/users/230eec29b1464006bb5d43c7ade31948 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:48.347642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81064d36-ae22-4cc2-af94-87e1de643923 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:48.348154 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-81064d36-ae22-4cc2-af94-87e1de643923 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:48.374489 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-66f9a1e6-b9d6-4471-9b21-dd94f4fb84f6 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:48.377024 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-66f9a1e6-b9d6-4471-9b21-dd94f4fb84f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:48.463183 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:48.469152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81064d36-ae22-4cc2-af94-87e1de643923 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:48.478907 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4491152-b232-4b93-aed5-e46b114c9bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:48.480136 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4491152-b232-4b93-aed5-e46b114c9bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:48.480586 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4491152-b232-4b93-aed5-e46b114c9bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:48.480872 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4491152-b232-4b93-aed5-e46b114c9bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:48.482285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4491152-b232-4b93-aed5-e46b114c9bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:48.484584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4491152-b232-4b93-aed5-e46b114c9bc6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:48.500389 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 920/1833] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:48.505219 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4491152-b232-4b93-aed5-e46b114c9bc6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:48.518832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4491152-b232-4b93-aed5-e46b114c9bc6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:48.532809 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 914/1834] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:48.534345 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209886f-db95-489a-b246-8f800583aeec None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:48.534741 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209886f-db95-489a-b246-8f800583aeec None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:48.536290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a209886f-db95-489a-b246-8f800583aeec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:48.566989 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc981c0-af7e-4b07-9e83-817f3a2bc5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:48.568004 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc981c0-af7e-4b07-9e83-817f3a2bc5ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:48.568293 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc981c0-af7e-4b07-9e83-817f3a2bc5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:48.568986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc981c0-af7e-4b07-9e83-817f3a2bc5ae admin admin] PATH_INFO: `/v3/projects/31d98c906480468b830fee7c37d725a4/users/7585ec5b962746fd98f07d5b45b6be96/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:48.593508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a209886f-db95-489a-b246-8f800583aeec None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:48.594855 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a209886f-db95-489a-b246-8f800583aeec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:48.616336 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 915/1835] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:48.616876 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc981c0-af7e-4b07-9e83-817f3a2bc5ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=31d98c906480468b830fee7c37d725a4, user_id=7585ec5b962746fd98f07d5b45b6be96, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:48.620625 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a209886f-db95-489a-b246-8f800583aeec None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:48.622867 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc981c0-af7e-4b07-9e83-817f3a2bc5ae admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:48.638550 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a209886f-db95-489a-b246-8f800583aeec None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:48.648336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 921/1836] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:48 2022] PUT /identity/v3/projects/31d98c906480468b830fee7c37d725a4/users/7585ec5b962746fd98f07d5b45b6be96/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:48.658902 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609813c-3535-452b-a044-5ad92bbe2988 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:48.659196 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609813c-3535-452b-a044-5ad92bbe2988 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:48.659482 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609813c-3535-452b-a044-5ad92bbe2988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:48.742036 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c609813c-3535-452b-a044-5ad92bbe2988 None None] MFA Rules not processed for user `7585ec5b962746fd98f07d5b45b6be96`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:48.743354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c609813c-3535-452b-a044-5ad92bbe2988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:48.763567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74302c52-f122-4a39-ae0d-e454a15865d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:48.764575 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74302c52-f122-4a39-ae0d-e454a15865d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:48.764862 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74302c52-f122-4a39-ae0d-e454a15865d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:48.765144 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74302c52-f122-4a39-ae0d-e454a15865d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:48.765943 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74302c52-f122-4a39-ae0d-e454a15865d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:33:48.766287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74302c52-f122-4a39-ae0d-e454a15865d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:48.772191 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74302c52-f122-4a39-ae0d-e454a15865d1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:48.810835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 916/1837] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:33:48.810835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549cf99-d80e-4638-9b11-44a46c9b790c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:48.810835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549cf99-d80e-4638-9b11-44a46c9b790c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:48.810835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549cf99-d80e-4638-9b11-44a46c9b790c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:48.813524 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 917/1838] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jun 29 12:33:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:33:48.814321 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549cf99-d80e-4638-9b11-44a46c9b790c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:48.864672 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549cf99-d80e-4638-9b11-44a46c9b790c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:48.889055 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc8ed0a-fe65-4fd2-9926-25032a36fce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:48.891166 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8ed0a-fe65-4fd2-9926-25032a36fce1 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:48.892349 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8ed0a-fe65-4fd2-9926-25032a36fce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:48.892802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc8ed0a-fe65-4fd2-9926-25032a36fce1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:48.896253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc8ed0a-fe65-4fd2-9926-25032a36fce1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:48.898713 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc8ed0a-fe65-4fd2-9926-25032a36fce1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:48.946479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 922/1839] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:48 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:33:48.952061 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c609813c-3535-452b-a044-5ad92bbe2988 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:48.959946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c609813c-3535-452b-a044-5ad92bbe2988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:48.976914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 918/1840] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:48 2022] POST /identity/v3/projects => generated 365 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:48.978903 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0548924-d275-4dd4-b927-6f82a8e5719d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:48.979349 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0548924-d275-4dd4-b927-6f82a8e5719d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:48.979774 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0548924-d275-4dd4-b927-6f82a8e5719d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:48.999602 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c609813c-3535-452b-a044-5ad92bbe2988 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:49.023233 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:49.047486 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c0548924-d275-4dd4-b927-6f82a8e5719d None None] MFA Rules not processed for user `3ac7391a3c41499d94e5f97478797313`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:49.048880 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c0548924-d275-4dd4-b927-6f82a8e5719d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:49.259295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 919/1841] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jun 29 12:33:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:49.260224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0548924-d275-4dd4-b927-6f82a8e5719d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:49.272609 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275cee4-0fee-4dc6-a735-403bd34fcaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:49.274116 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275cee4-0fee-4dc6-a735-403bd34fcaf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:49.274553 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275cee4-0fee-4dc6-a735-403bd34fcaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:49.275057 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9275cee4-0fee-4dc6-a735-403bd34fcaf9 admin admin] PATH_INFO: `/v3/projects/2727758125054870aeff24637f71b1c1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:49.276600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9275cee4-0fee-4dc6-a735-403bd34fcaf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2727758125054870aeff24637f71b1c1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:49.279470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9275cee4-0fee-4dc6-a735-403bd34fcaf9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:49.371129 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:49.391441 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9275cee4-0fee-4dc6-a735-403bd34fcaf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2727758125054870aeff24637f71b1c1', 'request_id': 'req-9275cee4-0fee-4dc6-a735-403bd34fcaf9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:49.403336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9275cee4-0fee-4dc6-a735-403bd34fcaf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2727758125054870aeff24637f71b1c1', 'request_id': 'req-9275cee4-0fee-4dc6-a735-403bd34fcaf9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:49.417230 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9275cee4-0fee-4dc6-a735-403bd34fcaf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2727758125054870aeff24637f71b1c1', 'request_id': 'req-9275cee4-0fee-4dc6-a735-403bd34fcaf9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:49.430094 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0548924-d275-4dd4-b927-6f82a8e5719d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:49.445272 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9275cee4-0fee-4dc6-a735-403bd34fcaf9 admin admin] The token cache is being invalidate because project 2727758125054870aeff24637f71b1c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:33:49.447323 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9275cee4-0fee-4dc6-a735-403bd34fcaf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:49.448673 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9275cee4-0fee-4dc6-a735-403bd34fcaf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:49.452623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 923/1842] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:33:48 2022] DELETE /identity/v3/projects/2727758125054870aeff24637f71b1c1 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:49.459878 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275cee4-0fee-4dc6-a735-403bd34fcaf9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:49.467727 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9275cee4-0fee-4dc6-a735-403bd34fcaf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:49.564893 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7b8df-6f13-49db-9906-1a950ed82cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:49.565662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7b8df-6f13-49db-9906-1a950ed82cc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:49.565947 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7b8df-6f13-49db-9906-1a950ed82cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:49.566774 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7b8df-6f13-49db-9906-1a950ed82cc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:49.571620 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7b8df-6f13-49db-9906-1a950ed82cc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:49.571620 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7b8df-6f13-49db-9906-1a950ed82cc9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:49.621571 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9275cee4-0fee-4dc6-a735-403bd34fcaf9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:49.629946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176d3b7-4a2b-499e-8ee4-f89aeb130e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:49.630682 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176d3b7-4a2b-499e-8ee4-f89aeb130e87 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:49.631675 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176d3b7-4a2b-499e-8ee4-f89aeb130e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:49.631675 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176d3b7-4a2b-499e-8ee4-f89aeb130e87 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:49.631950 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1176d3b7-4a2b-499e-8ee4-f89aeb130e87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:49.632773 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 920/1843] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:49 2022] POST /identity/v3/projects => generated 367 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:49.634954 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1176d3b7-4a2b-499e-8ee4-f89aeb130e87 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:49.635891 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce3a98-1429-4f48-8bba-d84286912abc None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:49.636330 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce3a98-1429-4f48-8bba-d84286912abc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:49.636749 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce3a98-1429-4f48-8bba-d84286912abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:49.682304 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 924/1844] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:49 2022] POST /identity/v3/users => generated 283 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:49.682304 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176d3b7-4a2b-499e-8ee4-f89aeb130e87 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:49.685060 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-85ce3a98-1429-4f48-8bba-d84286912abc None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:49.687882 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-85ce3a98-1429-4f48-8bba-d84286912abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:49.691946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1176d3b7-4a2b-499e-8ee4-f89aeb130e87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:49.726217 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176d3b7-4a2b-499e-8ee4-f89aeb130e87 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:49.822933 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 921/1845] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:49.824347 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce3a98-1429-4f48-8bba-d84286912abc None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:49.830030 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:49.856274 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce3a98-1429-4f48-8bba-d84286912abc None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:49.862547 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131ec0a-e637-4e6d-8290-fc29890e8391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:49.864842 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131ec0a-e637-4e6d-8290-fc29890e8391 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:49.865369 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131ec0a-e637-4e6d-8290-fc29890e8391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:49.865894 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131ec0a-e637-4e6d-8290-fc29890e8391 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:49.866874 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131ec0a-e637-4e6d-8290-fc29890e8391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:49.868910 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131ec0a-e637-4e6d-8290-fc29890e8391 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:49.883471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4be2c6-4bcd-49ad-8a64-ba819504853a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:49.884278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4be2c6-4bcd-49ad-8a64-ba819504853a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:49.884567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4be2c6-4bcd-49ad-8a64-ba819504853a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:49.884844 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4be2c6-4bcd-49ad-8a64-ba819504853a admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:49.885542 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4be2c6-4bcd-49ad-8a64-ba819504853a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:49.887659 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4be2c6-4bcd-49ad-8a64-ba819504853a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:49.895880 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 922/1846] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:49.897196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131ec0a-e637-4e6d-8290-fc29890e8391 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:49.899369 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f131ec0a-e637-4e6d-8290-fc29890e8391 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:49.928453 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 925/1847] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:49 2022] POST /identity/v3/users => generated 290 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:49.933826 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa82107b-14fc-4e45-ad98-8f0fb6ac5292 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:49.934172 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa82107b-14fc-4e45-ad98-8f0fb6ac5292 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:49.934487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa82107b-14fc-4e45-ad98-8f0fb6ac5292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:49.980128 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fa82107b-14fc-4e45-ad98-8f0fb6ac5292 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:49.984827 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fa82107b-14fc-4e45-ad98-8f0fb6ac5292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:50.009381 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 926/1848] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:50.010460 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa82107b-14fc-4e45-ad98-8f0fb6ac5292 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:50.011925 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:50.028312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa82107b-14fc-4e45-ad98-8f0fb6ac5292 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:50.041752 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e55190c-4778-4a87-8914-a1521b8cf999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:50.043054 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55190c-4778-4a87-8914-a1521b8cf999 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:50.043412 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55190c-4778-4a87-8914-a1521b8cf999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:50.043737 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e55190c-4778-4a87-8914-a1521b8cf999 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:50.044819 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e55190c-4778-4a87-8914-a1521b8cf999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:50.046874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e55190c-4778-4a87-8914-a1521b8cf999 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:50.054798 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131ec0a-e637-4e6d-8290-fc29890e8391 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:50.061204 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f15355-6f67-4f3d-bfd7-a380f3648aab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:50.061891 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f15355-6f67-4f3d-bfd7-a380f3648aab service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:50.062185 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f15355-6f67-4f3d-bfd7-a380f3648aab service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:50.062510 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f15355-6f67-4f3d-bfd7-a380f3648aab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:50.064184 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 927/1849] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:50.065053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e55190c-4778-4a87-8914-a1521b8cf999 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:50.068124 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a0f15355-6f67-4f3d-bfd7-a380f3648aab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:50.081586 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e55190c-4778-4a87-8914-a1521b8cf999 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:50.087475 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b5223-a9a1-4219-a3e6-0bef365b6371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:50.088160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b5223-a9a1-4219-a3e6-0bef365b6371 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:50.088465 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b5223-a9a1-4219-a3e6-0bef365b6371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:50.089468 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b5223-a9a1-4219-a3e6-0bef365b6371 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:50.089657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b5223-a9a1-4219-a3e6-0bef365b6371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:50.091674 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b5223-a9a1-4219-a3e6-0bef365b6371 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:50.097188 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f15355-6f67-4f3d-bfd7-a380f3648aab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:50.100565 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f15355-6f67-4f3d-bfd7-a380f3648aab service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:50.109687 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 928/1850] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:50.109687 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b5223-a9a1-4219-a3e6-0bef365b6371 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:50.118228 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 923/1851] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:50.119428 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f15355-6f67-4f3d-bfd7-a380f3648aab service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:50.121037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b5223-a9a1-4219-a3e6-0bef365b6371 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:50.133886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f15355-6f67-4f3d-bfd7-a380f3648aab service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:50.134502 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda469e-e30b-453b-bb5c-df46b85f86f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:50.135551 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda469e-e30b-453b-bb5c-df46b85f86f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:50.135843 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda469e-e30b-453b-bb5c-df46b85f86f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:50.136134 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fda469e-e30b-453b-bb5c-df46b85f86f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:50.138466 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fda469e-e30b-453b-bb5c-df46b85f86f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:33:50.138466 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fda469e-e30b-453b-bb5c-df46b85f86f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:50.142876 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3a7391-2f23-4bd2-a591-2bdb09c168e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:50.144065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a7391-2f23-4bd2-a591-2bdb09c168e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:50.145574 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a7391-2f23-4bd2-a591-2bdb09c168e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:50.146072 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a7391-2f23-4bd2-a591-2bdb09c168e1 admin admin] PATH_INFO: `/v3/projects/bde71ccc24074bc5903dd3a36a6a9816/users/2daec005681b4eb4a40f80fb426f7a50/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:50.169499 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3a7391-2f23-4bd2-a591-2bdb09c168e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bde71ccc24074bc5903dd3a36a6a9816, user_id=2daec005681b4eb4a40f80fb426f7a50, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:50.169623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fda469e-e30b-453b-bb5c-df46b85f86f9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:50.173010 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3a7391-2f23-4bd2-a591-2bdb09c168e1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:50.193497 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 924/1852] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:50 2022] PUT /identity/v3/projects/bde71ccc24074bc5903dd3a36a6a9816/users/2daec005681b4eb4a40f80fb426f7a50/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:50.193724 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 929/1853] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:33:50.194709 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda469e-e30b-453b-bb5c-df46b85f86f9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:50.198259 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3a7391-2f23-4bd2-a591-2bdb09c168e1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:50.225349 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fda469e-e30b-453b-bb5c-df46b85f86f9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:50.298205 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3a7391-2f23-4bd2-a591-2bdb09c168e1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:50.305025 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c6cb1-e849-43bd-9c38-716ab5ca54bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:50.305960 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c6cb1-e849-43bd-9c38-716ab5ca54bf admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:50.306598 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c6cb1-e849-43bd-9c38-716ab5ca54bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:50.307356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c6cb1-e849-43bd-9c38-716ab5ca54bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:50.309230 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3c6cb1-e849-43bd-9c38-716ab5ca54bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:50.315106 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3c6cb1-e849-43bd-9c38-716ab5ca54bf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:50.354100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97f61d-febc-4417-bb17-e0c586254700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:50.355357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97f61d-febc-4417-bb17-e0c586254700 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:50.355637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97f61d-febc-4417-bb17-e0c586254700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:50.355959 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd97f61d-febc-4417-bb17-e0c586254700 admin admin] PATH_INFO: `/v3/projects/f119a28bad9944ffa6ae45a86b328ca8/users/9b16af7a836b4a77844e0b186f264b93/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:50.385894 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 925/1854] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:50 2022] POST /identity/v3/projects => generated 371 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:50.386950 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba8b2a-99fd-41ad-8521-848944308d4c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:50.387279 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba8b2a-99fd-41ad-8521-848944308d4c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:50.387669 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ba8b2a-99fd-41ad-8521-848944308d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:50.410688 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd97f61d-febc-4417-bb17-e0c586254700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f119a28bad9944ffa6ae45a86b328ca8, user_id=9b16af7a836b4a77844e0b186f264b93, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:50.413632 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd97f61d-febc-4417-bb17-e0c586254700 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:50.460156 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 930/1855] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:50 2022] PUT /identity/v3/projects/f119a28bad9944ffa6ae45a86b328ca8/users/9b16af7a836b4a77844e0b186f264b93/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:50.461730 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97f61d-febc-4417-bb17-e0c586254700 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:50.471151 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-58ba8b2a-99fd-41ad-8521-848944308d4c None None] MFA Rules not processed for user `2daec005681b4eb4a40f80fb426f7a50`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:50.474574 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-58ba8b2a-99fd-41ad-8521-848944308d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:50.610159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 926/1856] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:50 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:50.610907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba8b2a-99fd-41ad-8521-848944308d4c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:50.639952 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd97f61d-febc-4417-bb17-e0c586254700 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:50.654260 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4dce6-7c62-4b56-84be-d567bfd07e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:50.656215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4dce6-7c62-4b56-84be-d567bfd07e41 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:50.656778 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4dce6-7c62-4b56-84be-d567bfd07e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:50.658227 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4dce6-7c62-4b56-84be-d567bfd07e41 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:50.660754 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:50.661456 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4dce6-7c62-4b56-84be-d567bfd07e41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:50.669676 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4dce6-7c62-4b56-84be-d567bfd07e41 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:50.695344 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:50.697200 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ba8b2a-99fd-41ad-8521-848944308d4c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:50.701446 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a1b7e-4696-4b98-85b4-d1c7719ec525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:50.702332 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a1b7e-4696-4b98-85b4-d1c7719ec525 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:50.702689 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a1b7e-4696-4b98-85b4-d1c7719ec525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:50.703037 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a1b7e-4696-4b98-85b4-d1c7719ec525 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:50.703869 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a1b7e-4696-4b98-85b4-d1c7719ec525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:50.705714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a1b7e-4696-4b98-85b4-d1c7719ec525 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:50.729112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 931/1857] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:50 2022] POST /identity/v3/users => generated 286 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:50.730002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4dce6-7c62-4b56-84be-d567bfd07e41 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:50.754185 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4dce6-7c62-4b56-84be-d567bfd07e41 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:50.765641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 927/1858] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:50.768874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d8be5-d95e-4cda-b0b4-8f58dbd1590d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:50.771267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d8be5-d95e-4cda-b0b4-8f58dbd1590d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:50.771842 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d8be5-d95e-4cda-b0b4-8f58dbd1590d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:50.771975 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a1b7e-4696-4b98-85b4-d1c7719ec525 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:50.772916 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d8be5-d95e-4cda-b0b4-8f58dbd1590d admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:50.775450 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605d8be5-d95e-4cda-b0b4-8f58dbd1590d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:50.778112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605d8be5-d95e-4cda-b0b4-8f58dbd1590d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:50.798876 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a1b7e-4696-4b98-85b4-d1c7719ec525 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:50.821804 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e640719-12bf-4a97-aaaf-39cce7de273f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:50.826576 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e640719-12bf-4a97-aaaf-39cce7de273f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:50.826959 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e640719-12bf-4a97-aaaf-39cce7de273f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:50.827210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e640719-12bf-4a97-aaaf-39cce7de273f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:50.828117 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e640719-12bf-4a97-aaaf-39cce7de273f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:50.837769 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e640719-12bf-4a97-aaaf-39cce7de273f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:50.853387 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 932/1859] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:50 2022] POST /identity/v3/projects => generated 363 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:50.854577 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d8be5-d95e-4cda-b0b4-8f58dbd1590d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:50.868406 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 928/1860] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:50 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:50.869315 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e640719-12bf-4a97-aaaf-39cce7de273f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:50.890433 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e640719-12bf-4a97-aaaf-39cce7de273f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:50.961432 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d8be5-d95e-4cda-b0b4-8f58dbd1590d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:50.968985 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29929b6a-8983-4b6f-88a3-c3d1ac2b5839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:50.969911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29929b6a-8983-4b6f-88a3-c3d1ac2b5839 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:50.970911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29929b6a-8983-4b6f-88a3-c3d1ac2b5839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:50.971204 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29929b6a-8983-4b6f-88a3-c3d1ac2b5839 admin admin] PATH_INFO: `/v3/projects/f119a28bad9944ffa6ae45a86b328ca8/users/9b16af7a836b4a77844e0b186f264b93/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:50.975453 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29929b6a-8983-4b6f-88a3-c3d1ac2b5839 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f119a28bad9944ffa6ae45a86b328ca8, user_id=9b16af7a836b4a77844e0b186f264b93, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:50.981100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29929b6a-8983-4b6f-88a3-c3d1ac2b5839 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:51.004572 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 933/1861] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:50 2022] PUT /identity/v3/projects/f119a28bad9944ffa6ae45a86b328ca8/users/9b16af7a836b4a77844e0b186f264b93/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:51.005751 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29929b6a-8983-4b6f-88a3-c3d1ac2b5839 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:51.053529 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50d1c6-39e1-4b94-ac94-6ee6796b85e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:51.054616 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50d1c6-39e1-4b94-ac94-6ee6796b85e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:51.055000 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50d1c6-39e1-4b94-ac94-6ee6796b85e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:51.055701 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50d1c6-39e1-4b94-ac94-6ee6796b85e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:51.056838 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50d1c6-39e1-4b94-ac94-6ee6796b85e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:51.059803 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50d1c6-39e1-4b94-ac94-6ee6796b85e7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:51.107475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 929/1862] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:50 2022] POST /identity/v3/users => generated 288 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:51.108875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c66120-fd4e-4349-8f72-94e17104826e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:51.109166 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c66120-fd4e-4349-8f72-94e17104826e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:51.109447 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c66120-fd4e-4349-8f72-94e17104826e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:51.129698 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29929b6a-8983-4b6f-88a3-c3d1ac2b5839 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:51.137107 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714b993-8672-4edb-a6cd-f804d13ec2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:51.138266 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714b993-8672-4edb-a6cd-f804d13ec2cb admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:51.138776 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714b993-8672-4edb-a6cd-f804d13ec2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:51.139283 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714b993-8672-4edb-a6cd-f804d13ec2cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:51.140954 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b714b993-8672-4edb-a6cd-f804d13ec2cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:51.144812 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b714b993-8672-4edb-a6cd-f804d13ec2cb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:51.160956 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 934/1863] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:51.166362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714b993-8672-4edb-a6cd-f804d13ec2cb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:51.177979 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714b993-8672-4edb-a6cd-f804d13ec2cb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:51.182068 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-22c66120-fd4e-4349-8f72-94e17104826e None None] MFA Rules not processed for user `9b16af7a836b4a77844e0b186f264b93`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:51.182998 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a857d-1208-4b86-88a0-1ee427bc5b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:51.183333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-22c66120-fd4e-4349-8f72-94e17104826e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:51.183686 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a857d-1208-4b86-88a0-1ee427bc5b5b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:51.183990 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a857d-1208-4b86-88a0-1ee427bc5b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:51.184257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a857d-1208-4b86-88a0-1ee427bc5b5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:51.184999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441a857d-1208-4b86-88a0-1ee427bc5b5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:51.187115 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441a857d-1208-4b86-88a0-1ee427bc5b5b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:51.199254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 935/1864] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:51.200183 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a857d-1208-4b86-88a0-1ee427bc5b5b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:51.211847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441a857d-1208-4b86-88a0-1ee427bc5b5b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:51.222768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4032b9-b071-476e-8227-6c159a1f00fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:51.225384 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4032b9-b071-476e-8227-6c159a1f00fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:51.229456 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4032b9-b071-476e-8227-6c159a1f00fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:51.229897 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4032b9-b071-476e-8227-6c159a1f00fd admin admin] PATH_INFO: `/v3/projects/cdf8f2e014e64ec8bc1986d3b7f119f6/users/28a56a07ace24cc48931ed01ffde2db1/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:51.276399 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4032b9-b071-476e-8227-6c159a1f00fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdf8f2e014e64ec8bc1986d3b7f119f6, user_id=28a56a07ace24cc48931ed01ffde2db1, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:51.288762 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4032b9-b071-476e-8227-6c159a1f00fd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:51.316871 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 936/1865] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:51 2022] PUT /identity/v3/projects/cdf8f2e014e64ec8bc1986d3b7f119f6/users/28a56a07ace24cc48931ed01ffde2db1/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:51.317828 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4032b9-b071-476e-8227-6c159a1f00fd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:51.368023 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 930/1866] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:51 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:51.370614 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a243d7-1605-4d3b-9f10-714a3cae3e63 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:51.371072 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a243d7-1605-4d3b-9f10-714a3cae3e63 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:51.371531 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a243d7-1605-4d3b-9f10-714a3cae3e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:51.427353 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:51.434523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f6a243d7-1605-4d3b-9f10-714a3cae3e63 None None] MFA Rules not processed for user `28a56a07ace24cc48931ed01ffde2db1`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:51.436033 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f6a243d7-1605-4d3b-9f10-714a3cae3e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:51.471015 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:51.514354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4032b9-b071-476e-8227-6c159a1f00fd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:51.531394 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3dd6fe-bb01-4e84-8157-85ae86cc91e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:51.533953 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dd6fe-bb01-4e84-8157-85ae86cc91e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:51.533953 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dd6fe-bb01-4e84-8157-85ae86cc91e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:51.533953 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3dd6fe-bb01-4e84-8157-85ae86cc91e3 admin admin] PATH_INFO: `/v3/projects/92cd54a9a5524d04b5074fde9d8e8ac2/users/9d8b6be7d45748b5a6c049b6352c0392/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:51.559024 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 931/1867] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:51.564671 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a243d7-1605-4d3b-9f10-714a3cae3e63 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:51.567007 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f6a243d7-1605-4d3b-9f10-714a3cae3e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:51.603307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3dd6fe-bb01-4e84-8157-85ae86cc91e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92cd54a9a5524d04b5074fde9d8e8ac2, user_id=9d8b6be7d45748b5a6c049b6352c0392, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:51.606150 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3dd6fe-bb01-4e84-8157-85ae86cc91e3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:51.626131 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 937/1868] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:51 2022] PUT /identity/v3/projects/92cd54a9a5524d04b5074fde9d8e8ac2/users/9d8b6be7d45748b5a6c049b6352c0392/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:51.627244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3dd6fe-bb01-4e84-8157-85ae86cc91e3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:51.630601 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3a3dd6fe-bb01-4e84-8157-85ae86cc91e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:51.685263 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a243d7-1605-4d3b-9f10-714a3cae3e63 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:51.709206 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3dd6fe-bb01-4e84-8157-85ae86cc91e3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:51.720363 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6261ec-8dc2-437d-b1a5-c2dd446aea4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:51.721037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6261ec-8dc2-437d-b1a5-c2dd446aea4d service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:51.721325 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6261ec-8dc2-437d-b1a5-c2dd446aea4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:51.721628 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6261ec-8dc2-437d-b1a5-c2dd446aea4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:51.750931 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bffe44e-d396-44c6-b657-ec01da228239 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:51.751605 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffe44e-d396-44c6-b657-ec01da228239 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:51.751871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffe44e-d396-44c6-b657-ec01da228239 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:51.752151 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffe44e-d396-44c6-b657-ec01da228239 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:51.786755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6261ec-8dc2-437d-b1a5-c2dd446aea4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:51.788191 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6261ec-8dc2-437d-b1a5-c2dd446aea4d service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:51.797556 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 938/1869] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:51.798494 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6261ec-8dc2-437d-b1a5-c2dd446aea4d service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:51.825001 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bffe44e-d396-44c6-b657-ec01da228239 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:51.826641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bffe44e-d396-44c6-b657-ec01da228239 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:51.843432 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 932/1870] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:51 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:51.844503 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bffe44e-d396-44c6-b657-ec01da228239 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:51.923867 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:51.943619 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6261ec-8dc2-437d-b1a5-c2dd446aea4d service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:51.952518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e57082-860a-4cc4-b33c-d954874304b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:51.953209 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e57082-860a-4cc4-b33c-d954874304b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:51.953499 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e57082-860a-4cc4-b33c-d954874304b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:51.953782 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e57082-860a-4cc4-b33c-d954874304b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:51.954635 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e57082-860a-4cc4-b33c-d954874304b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:51.956440 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e57082-860a-4cc4-b33c-d954874304b6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:51.978729 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 939/1871] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:51.979661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e57082-860a-4cc4-b33c-d954874304b6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:51.987130 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:51.992854 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e57082-860a-4cc4-b33c-d954874304b6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:51.996903 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92ca28-ae3f-494c-9528-ad1b83bd3513 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:51.997756 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92ca28-ae3f-494c-9528-ad1b83bd3513 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:51.998569 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92ca28-ae3f-494c-9528-ad1b83bd3513 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:51.999242 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92ca28-ae3f-494c-9528-ad1b83bd3513 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:52.002981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3f92ca28-ae3f-494c-9528-ad1b83bd3513 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:52.005407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bffe44e-d396-44c6-b657-ec01da228239 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:52.011873 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aef9019-310d-4e2a-bf15-254217efe90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:52.013346 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef9019-310d-4e2a-bf15-254217efe90e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:52.013641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef9019-310d-4e2a-bf15-254217efe90e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:52.013922 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aef9019-310d-4e2a-bf15-254217efe90e admin admin] PATH_INFO: `/v3/users/28a56a07ace24cc48931ed01ffde2db1` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:52.018659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef9019-310d-4e2a-bf15-254217efe90e admin admin] RBAC: Authorizing `identity:delete_user(user_id=28a56a07ace24cc48931ed01ffde2db1)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:52.026934 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aef9019-310d-4e2a-bf15-254217efe90e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:52.094180 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f92ca28-ae3f-494c-9528-ad1b83bd3513 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:52.096108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f92ca28-ae3f-494c-9528-ad1b83bd3513 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:52.118230 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1aef9019-310d-4e2a-bf15-254217efe90e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28a56a07ace24cc48931ed01ffde2db1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:52.122922 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 940/1872] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:51 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:52.126683 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92ca28-ae3f-494c-9528-ad1b83bd3513 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:52.130935 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1aef9019-310d-4e2a-bf15-254217efe90e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28a56a07ace24cc48931ed01ffde2db1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:52.147041 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92ca28-ae3f-494c-9528-ad1b83bd3513 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:52.155487 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1aef9019-310d-4e2a-bf15-254217efe90e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28a56a07ace24cc48931ed01ffde2db1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:52.166890 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1aef9019-310d-4e2a-bf15-254217efe90e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28a56a07ace24cc48931ed01ffde2db1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:52.167858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1aef9019-310d-4e2a-bf15-254217efe90e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28a56a07ace24cc48931ed01ffde2db1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:52.171723 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c7e35-df63-4d4a-a8f2-51919dd4c078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:52.172638 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c7e35-df63-4d4a-a8f2-51919dd4c078 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:52.172918 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c7e35-df63-4d4a-a8f2-51919dd4c078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:52.173192 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c7e35-df63-4d4a-a8f2-51919dd4c078 admin admin] PATH_INFO: `/v3/projects/92cd54a9a5524d04b5074fde9d8e8ac2/users/9d8b6be7d45748b5a6c049b6352c0392/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:52.179498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c7e35-df63-4d4a-a8f2-51919dd4c078 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92cd54a9a5524d04b5074fde9d8e8ac2, user_id=9d8b6be7d45748b5a6c049b6352c0392, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:52.183619 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c7e35-df63-4d4a-a8f2-51919dd4c078 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:52.194291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 933/1873] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:33:51 2022] DELETE /identity/v3/users/28a56a07ace24cc48931ed01ffde2db1 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:52.209245 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ef069-366f-4c06-baf1-f986c3dd01df None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:52.209707 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ef069-366f-4c06-baf1-f986c3dd01df None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:52.211008 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ef069-366f-4c06-baf1-f986c3dd01df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:52.221693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 941/1874] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:52 2022] PUT /identity/v3/projects/92cd54a9a5524d04b5074fde9d8e8ac2/users/9d8b6be7d45748b5a6c049b6352c0392/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:52.236766 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d08ef069-366f-4c06-baf1-f986c3dd01df None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:52.239206 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d08ef069-366f-4c06-baf1-f986c3dd01df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:52.242503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1713f-c3ee-4845-817e-efa911161162 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:52.244095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1713f-c3ee-4845-817e-efa911161162 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:52.244095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f1713f-c3ee-4845-817e-efa911161162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:52.324775 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a8f1713f-c3ee-4845-817e-efa911161162 None None] MFA Rules not processed for user `9d8b6be7d45748b5a6c049b6352c0392`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:52.325999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a8f1713f-c3ee-4845-817e-efa911161162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:52.430331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 934/1875] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:52.447388 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ef069-366f-4c06-baf1-f986c3dd01df None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:52.448865 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d08ef069-366f-4c06-baf1-f986c3dd01df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:52.605272 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 942/1876] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:52 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:52.618306 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f1713f-c3ee-4845-817e-efa911161162 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:52.621789 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a8f1713f-c3ee-4845-817e-efa911161162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:52.629676 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ef069-366f-4c06-baf1-f986c3dd01df None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:52.634003 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bebd2e-bddd-49db-8117-372e504f0a67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:52.634954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bebd2e-bddd-49db-8117-372e504f0a67 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:52.635290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bebd2e-bddd-49db-8117-372e504f0a67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:52.635809 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bebd2e-bddd-49db-8117-372e504f0a67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:52.646920 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:52.663820 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bebd2e-bddd-49db-8117-372e504f0a67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:52.666112 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bebd2e-bddd-49db-8117-372e504f0a67 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:52.702246 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 935/1877] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:52.769919 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f1713f-c3ee-4845-817e-efa911161162 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:52.774644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eaf9c2-8c4b-40ef-bb2a-bb949d0b3882 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:52.775321 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eaf9c2-8c4b-40ef-bb2a-bb949d0b3882 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:52.775612 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eaf9c2-8c4b-40ef-bb2a-bb949d0b3882 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:52.775894 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eaf9c2-8c4b-40ef-bb2a-bb949d0b3882 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:52.788108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eaf9c2-8c4b-40ef-bb2a-bb949d0b3882 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:52.789641 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eaf9c2-8c4b-40ef-bb2a-bb949d0b3882 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:52.814628 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 943/1878] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:52 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:53.196855 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bebd2e-bddd-49db-8117-372e504f0a67 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:53.201532 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-79bebd2e-bddd-49db-8117-372e504f0a67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:53.241950 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bebd2e-bddd-49db-8117-372e504f0a67 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:53.247009 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c38e56-deb3-406b-94ef-2a1b0799db34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:53.248323 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c38e56-deb3-406b-94ef-2a1b0799db34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:53.248499 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c38e56-deb3-406b-94ef-2a1b0799db34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:53.248767 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c38e56-deb3-406b-94ef-2a1b0799db34 admin admin] PATH_INFO: `/v3/projects/cdf8f2e014e64ec8bc1986d3b7f119f6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:53.250038 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c38e56-deb3-406b-94ef-2a1b0799db34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdf8f2e014e64ec8bc1986d3b7f119f6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:53.258818 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c38e56-deb3-406b-94ef-2a1b0799db34 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:53.348487 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b1c38e56-deb3-406b-94ef-2a1b0799db34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdf8f2e014e64ec8bc1986d3b7f119f6', 'request_id': 'req-b1c38e56-deb3-406b-94ef-2a1b0799db34'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:53.355862 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b1c38e56-deb3-406b-94ef-2a1b0799db34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdf8f2e014e64ec8bc1986d3b7f119f6', 'request_id': 'req-b1c38e56-deb3-406b-94ef-2a1b0799db34'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:53.368666 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b1c38e56-deb3-406b-94ef-2a1b0799db34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdf8f2e014e64ec8bc1986d3b7f119f6', 'request_id': 'req-b1c38e56-deb3-406b-94ef-2a1b0799db34'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:53.399994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b1c38e56-deb3-406b-94ef-2a1b0799db34 admin admin] The token cache is being invalidate because project cdf8f2e014e64ec8bc1986d3b7f119f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:33:53.400462 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b1c38e56-deb3-406b-94ef-2a1b0799db34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:53.402582 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b1c38e56-deb3-406b-94ef-2a1b0799db34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:53.418543 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 936/1879] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:33:53 2022] DELETE /identity/v3/projects/cdf8f2e014e64ec8bc1986d3b7f119f6 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:53.419600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad36de-7bde-4299-a6ca-a31d0c376425 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:53.420050 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad36de-7bde-4299-a6ca-a31d0c376425 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:53.420230 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad36de-7bde-4299-a6ca-a31d0c376425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:53.439261 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c38e56-deb3-406b-94ef-2a1b0799db34 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:53.445328 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4fad36de-7bde-4299-a6ca-a31d0c376425 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:53.446375 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b1c38e56-deb3-406b-94ef-2a1b0799db34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:53.446825 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4fad36de-7bde-4299-a6ca-a31d0c376425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:53.478041 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:53.557311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c38e56-deb3-406b-94ef-2a1b0799db34 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:53.562961 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae75f9-7d04-4359-8a4a-b866301d1fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:53.564214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae75f9-7d04-4359-8a4a-b866301d1fc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:53.564602 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae75f9-7d04-4359-8a4a-b866301d1fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:53.565080 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae75f9-7d04-4359-8a4a-b866301d1fc1 admin admin] PATH_INFO: `/v3/users/2daec005681b4eb4a40f80fb426f7a50` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:53.567220 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae75f9-7d04-4359-8a4a-b866301d1fc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2daec005681b4eb4a40f80fb426f7a50)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:53.569427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae75f9-7d04-4359-8a4a-b866301d1fc1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:53.575675 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 944/1880] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:33:53.583258 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183a08e-ecdb-4d8c-88d6-6191e5152747 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:53.583575 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183a08e-ecdb-4d8c-88d6-6191e5152747 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:53.583834 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0183a08e-ecdb-4d8c-88d6-6191e5152747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:53.603916 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0183a08e-ecdb-4d8c-88d6-6191e5152747 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:53.605131 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0183a08e-ecdb-4d8c-88d6-6191e5152747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:53.627098 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 945/1881] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:33:53.632419 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0183a08e-ecdb-4d8c-88d6-6191e5152747 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:53.642477 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0183a08e-ecdb-4d8c-88d6-6191e5152747 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:53.648454 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e553348-dcc6-4946-bf16-e3f6fe0e7b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:53.649131 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e553348-dcc6-4946-bf16-e3f6fe0e7b2e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:53.649415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e553348-dcc6-4946-bf16-e3f6fe0e7b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:53.649766 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e553348-dcc6-4946-bf16-e3f6fe0e7b2e admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:53.650547 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e553348-dcc6-4946-bf16-e3f6fe0e7b2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:33:53.651919 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e553348-dcc6-4946-bf16-e3f6fe0e7b2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:53.653551 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e553348-dcc6-4946-bf16-e3f6fe0e7b2e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:53.668924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5cae75f9-7d04-4359-8a4a-b866301d1fc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2daec005681b4eb4a40f80fb426f7a50'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:53.683124 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 946/1882] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:33:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:33:53.688215 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5cae75f9-7d04-4359-8a4a-b866301d1fc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2daec005681b4eb4a40f80fb426f7a50'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:53.692977 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e553348-dcc6-4946-bf16-e3f6fe0e7b2e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:53.705074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e553348-dcc6-4946-bf16-e3f6fe0e7b2e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:53.716026 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92e170-2691-46e7-85e4-08e2c93d1405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:53.717589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92e170-2691-46e7-85e4-08e2c93d1405 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:53.717884 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92e170-2691-46e7-85e4-08e2c93d1405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:53.718200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92e170-2691-46e7-85e4-08e2c93d1405 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:53.720754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92e170-2691-46e7-85e4-08e2c93d1405 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:53.729373 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92e170-2691-46e7-85e4-08e2c93d1405 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:53.729565 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5cae75f9-7d04-4359-8a4a-b866301d1fc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2daec005681b4eb4a40f80fb426f7a50'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:53.730311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5cae75f9-7d04-4359-8a4a-b866301d1fc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2daec005681b4eb4a40f80fb426f7a50'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:53.730977 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5cae75f9-7d04-4359-8a4a-b866301d1fc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2daec005681b4eb4a40f80fb426f7a50'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:53.744073 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:53.765342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 937/1883] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:33:53 2022] DELETE /identity/v3/users/2daec005681b4eb4a40f80fb426f7a50 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:53.790751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2173d4-edc0-4480-825c-714f49862ead None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:53.790751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2173d4-edc0-4480-825c-714f49862ead None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:53.790751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2173d4-edc0-4480-825c-714f49862ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:53.801030 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 947/1884] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:53 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:53.803556 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92e170-2691-46e7-85e4-08e2c93d1405 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:53.805233 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4f92e170-2691-46e7-85e4-08e2c93d1405 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:53.830049 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0a2173d4-edc0-4480-825c-714f49862ead None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:53.831307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0a2173d4-edc0-4480-825c-714f49862ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:53.912158 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92e170-2691-46e7-85e4-08e2c93d1405 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:53.918686 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a4bb0-d08a-4606-ae54-31d368d222ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:53.919559 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a4bb0-d08a-4606-ae54-31d368d222ad admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:53.919858 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a4bb0-d08a-4606-ae54-31d368d222ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:53.920207 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a4bb0-d08a-4606-ae54-31d368d222ad admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:53.921103 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a4bb0-d08a-4606-ae54-31d368d222ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:53.923142 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a4bb0-d08a-4606-ae54-31d368d222ad admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:53.946671 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 938/1885] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:53.955965 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 948/1886] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:53 2022] POST /identity/v3/users => generated 287 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:53.958273 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2173d4-edc0-4480-825c-714f49862ead None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:53.960756 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0a2173d4-edc0-4480-825c-714f49862ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:53.962653 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a4bb0-d08a-4606-ae54-31d368d222ad admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:53.982870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a4bb0-d08a-4606-ae54-31d368d222ad admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:53.999913 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212147b4-0506-4eb9-a881-f7dfbc077625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:54.001156 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212147b4-0506-4eb9-a881-f7dfbc077625 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:54.001383 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212147b4-0506-4eb9-a881-f7dfbc077625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:54.001730 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212147b4-0506-4eb9-a881-f7dfbc077625 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:54.002507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212147b4-0506-4eb9-a881-f7dfbc077625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:54.004803 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212147b4-0506-4eb9-a881-f7dfbc077625 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:54.026385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 949/1887] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:53 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:54.031901 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212147b4-0506-4eb9-a881-f7dfbc077625 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:54.049671 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2173d4-edc0-4480-825c-714f49862ead None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:54.057594 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e860329a-27cb-4c21-9195-f0067b999adc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:54.059041 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860329a-27cb-4c21-9195-f0067b999adc service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:54.059628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860329a-27cb-4c21-9195-f0067b999adc service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:54.059824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212147b4-0506-4eb9-a881-f7dfbc077625 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:54.060172 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e860329a-27cb-4c21-9195-f0067b999adc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:54.072774 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf89370-1dad-48e4-b158-a13f830c6901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:54.073637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf89370-1dad-48e4-b158-a13f830c6901 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:54.074043 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf89370-1dad-48e4-b158-a13f830c6901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:54.074389 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf89370-1dad-48e4-b158-a13f830c6901 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:54.075151 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf89370-1dad-48e4-b158-a13f830c6901 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:54.076751 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf89370-1dad-48e4-b158-a13f830c6901 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:54.082614 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e860329a-27cb-4c21-9195-f0067b999adc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:54.085548 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e860329a-27cb-4c21-9195-f0067b999adc service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:54.106348 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 950/1888] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:54.108203 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 939/1889] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 12:33:54.110654 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf89370-1dad-48e4-b158-a13f830c6901 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:54.127431 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf89370-1dad-48e4-b158-a13f830c6901 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:54.134446 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2b346-64bd-41a8-926b-14b480823352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:54.135544 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2b346-64bd-41a8-926b-14b480823352 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:54.135829 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2b346-64bd-41a8-926b-14b480823352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:54.136104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2b346-64bd-41a8-926b-14b480823352 admin admin] PATH_INFO: `/v3/projects/ed93385ff03f4672acc79e103ffcb82e/users/bba6a17fc19645c29fce6fa655095a0a/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:54.172310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2b346-64bd-41a8-926b-14b480823352 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed93385ff03f4672acc79e103ffcb82e, user_id=bba6a17fc19645c29fce6fa655095a0a, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:54.175907 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2b346-64bd-41a8-926b-14b480823352 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:54.200368 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 951/1890] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:54 2022] PUT /identity/v3/projects/ed93385ff03f4672acc79e103ffcb82e/users/bba6a17fc19645c29fce6fa655095a0a/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:54.205928 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ad5b3-a5e5-46ef-8482-b3475a12785d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:54.206257 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ad5b3-a5e5-46ef-8482-b3475a12785d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:54.207112 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ad5b3-a5e5-46ef-8482-b3475a12785d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:54.226641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f26ad5b3-a5e5-46ef-8482-b3475a12785d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:54.227884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f26ad5b3-a5e5-46ef-8482-b3475a12785d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:54.308729 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 940/1891] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:54.317959 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2b346-64bd-41a8-926b-14b480823352 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:54.370799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:54.419572 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2b346-64bd-41a8-926b-14b480823352 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:54.424151 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6cb03-3e14-4a99-a70f-03494a3affbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:54.424841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6cb03-3e14-4a99-a70f-03494a3affbe service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:54.426379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6cb03-3e14-4a99-a70f-03494a3affbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:54.426773 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6cb03-3e14-4a99-a70f-03494a3affbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:54.438129 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6cb03-3e14-4a99-a70f-03494a3affbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:54.439716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6cb03-3e14-4a99-a70f-03494a3affbe service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:54.451290 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 952/1892] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:54.495078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ad5b3-a5e5-46ef-8482-b3475a12785d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:54.497236 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f26ad5b3-a5e5-46ef-8482-b3475a12785d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:54.514654 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ad5b3-a5e5-46ef-8482-b3475a12785d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:54.519717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65938d9c-21cf-457c-9ac5-37aa085be65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:54.521164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65938d9c-21cf-457c-9ac5-37aa085be65f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:54.521429 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65938d9c-21cf-457c-9ac5-37aa085be65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:54.521716 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65938d9c-21cf-457c-9ac5-37aa085be65f admin admin] PATH_INFO: `/v3/projects/bde71ccc24074bc5903dd3a36a6a9816` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:54.523526 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65938d9c-21cf-457c-9ac5-37aa085be65f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bde71ccc24074bc5903dd3a36a6a9816)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:54.525489 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65938d9c-21cf-457c-9ac5-37aa085be65f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:54.607229 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:54.630060 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-65938d9c-21cf-457c-9ac5-37aa085be65f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bde71ccc24074bc5903dd3a36a6a9816', 'request_id': 'req-65938d9c-21cf-457c-9ac5-37aa085be65f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:54.638161 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-65938d9c-21cf-457c-9ac5-37aa085be65f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bde71ccc24074bc5903dd3a36a6a9816', 'request_id': 'req-65938d9c-21cf-457c-9ac5-37aa085be65f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:54.645758 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-65938d9c-21cf-457c-9ac5-37aa085be65f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bde71ccc24074bc5903dd3a36a6a9816', 'request_id': 'req-65938d9c-21cf-457c-9ac5-37aa085be65f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:54.662020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-65938d9c-21cf-457c-9ac5-37aa085be65f admin admin] The token cache is being invalidate because project bde71ccc24074bc5903dd3a36a6a9816 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:33:54.662331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-65938d9c-21cf-457c-9ac5-37aa085be65f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:54.663182 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-65938d9c-21cf-457c-9ac5-37aa085be65f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:54.666264 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 941/1893] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:33:54 2022] DELETE /identity/v3/projects/bde71ccc24074bc5903dd3a36a6a9816 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:54.670158 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6cb03-3e14-4a99-a70f-03494a3affbe service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:54.681180 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6cb03-3e14-4a99-a70f-03494a3affbe service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:54.783120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b159cb2-5f4e-431a-98af-e85d9329a82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:54.783762 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b159cb2-5f4e-431a-98af-e85d9329a82b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:54.784320 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b159cb2-5f4e-431a-98af-e85d9329a82b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:54.785408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b159cb2-5f4e-431a-98af-e85d9329a82b admin admin] PATH_INFO: `/v3/users/9d8b6be7d45748b5a6c049b6352c0392` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:54.787287 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b159cb2-5f4e-431a-98af-e85d9329a82b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d8b6be7d45748b5a6c049b6352c0392)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:54.789380 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b159cb2-5f4e-431a-98af-e85d9329a82b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:54.890664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0b159cb2-5f4e-431a-98af-e85d9329a82b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d8b6be7d45748b5a6c049b6352c0392'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:54.891572 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0b159cb2-5f4e-431a-98af-e85d9329a82b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d8b6be7d45748b5a6c049b6352c0392'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:54.916445 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0b159cb2-5f4e-431a-98af-e85d9329a82b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d8b6be7d45748b5a6c049b6352c0392'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:54.917407 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0b159cb2-5f4e-431a-98af-e85d9329a82b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d8b6be7d45748b5a6c049b6352c0392'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:54.941084 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0b159cb2-5f4e-431a-98af-e85d9329a82b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d8b6be7d45748b5a6c049b6352c0392'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:54.983779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 953/1894] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:33:54 2022] DELETE /identity/v3/users/9d8b6be7d45748b5a6c049b6352c0392 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:55.661908 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65938d9c-21cf-457c-9ac5-37aa085be65f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:55.663698 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-65938d9c-21cf-457c-9ac5-37aa085be65f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:55.686857 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65938d9c-21cf-457c-9ac5-37aa085be65f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:55.765003 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befaf4e-efda-4db0-9a94-1d79bc29e1ff None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:55.765558 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befaf4e-efda-4db0-9a94-1d79bc29e1ff None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:55.765558 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befaf4e-efda-4db0-9a94-1d79bc29e1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:55.810068 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ede736-b7fd-4db2-b9d4-608cd8595b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:55.811296 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ede736-b7fd-4db2-b9d4-608cd8595b07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:55.811749 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ede736-b7fd-4db2-b9d4-608cd8595b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:55.812176 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ede736-b7fd-4db2-b9d4-608cd8595b07 admin admin] PATH_INFO: `/v3/projects/92cd54a9a5524d04b5074fde9d8e8ac2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:55.813969 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ede736-b7fd-4db2-b9d4-608cd8595b07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92cd54a9a5524d04b5074fde9d8e8ac2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:55.815117 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0befaf4e-efda-4db0-9a94-1d79bc29e1ff None None] MFA Rules not processed for user `bba6a17fc19645c29fce6fa655095a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:55.817344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ede736-b7fd-4db2-b9d4-608cd8595b07 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:55.818958 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0befaf4e-efda-4db0-9a94-1d79bc29e1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:55.883299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:55.898330 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d1ede736-b7fd-4db2-b9d4-608cd8595b07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92cd54a9a5524d04b5074fde9d8e8ac2', 'request_id': 'req-d1ede736-b7fd-4db2-b9d4-608cd8595b07'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:55.910918 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d1ede736-b7fd-4db2-b9d4-608cd8595b07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92cd54a9a5524d04b5074fde9d8e8ac2', 'request_id': 'req-d1ede736-b7fd-4db2-b9d4-608cd8595b07'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:55.923939 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d1ede736-b7fd-4db2-b9d4-608cd8595b07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92cd54a9a5524d04b5074fde9d8e8ac2', 'request_id': 'req-d1ede736-b7fd-4db2-b9d4-608cd8595b07'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:55.951346 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d1ede736-b7fd-4db2-b9d4-608cd8595b07 admin admin] The token cache is being invalidate because project 92cd54a9a5524d04b5074fde9d8e8ac2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:33:55.951649 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d1ede736-b7fd-4db2-b9d4-608cd8595b07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:55.952353 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d1ede736-b7fd-4db2-b9d4-608cd8595b07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:55.954501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 942/1895] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:33:55 2022] DELETE /identity/v3/projects/92cd54a9a5524d04b5074fde9d8e8ac2 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:55.960494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2457f2-2f12-432d-bb5a-49f7c0f79c36 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:55.961134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2457f2-2f12-432d-bb5a-49f7c0f79c36 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:55.961532 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2457f2-2f12-432d-bb5a-49f7c0f79c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:55.989455 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 954/1896] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:55.992239 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ab2457f2-2f12-432d-bb5a-49f7c0f79c36 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:55.993657 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ab2457f2-2f12-432d-bb5a-49f7c0f79c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:56.001909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befaf4e-efda-4db0-9a94-1d79bc29e1ff None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:56.003437 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0befaf4e-efda-4db0-9a94-1d79bc29e1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:56.108375 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befaf4e-efda-4db0-9a94-1d79bc29e1ff None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:56.139253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:56.173316 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 943/1897] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:56.174911 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4da763-2940-4cf7-8619-8f40131f2e60 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:56.175207 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4da763-2940-4cf7-8619-8f40131f2e60 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:56.176059 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4da763-2940-4cf7-8619-8f40131f2e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:56.202617 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ea4da763-2940-4cf7-8619-8f40131f2e60 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:56.204130 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ea4da763-2940-4cf7-8619-8f40131f2e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:56.220330 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f2970-d947-464d-b453-f15c5920b609 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:56.221299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f2970-d947-464d-b453-f15c5920b609 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:56.223936 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 944/1898] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:56.227605 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f2970-d947-464d-b453-f15c5920b609 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:56.228065 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f2970-d947-464d-b453-f15c5920b609 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:56.230320 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4da763-2940-4cf7-8619-8f40131f2e60 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:56.248855 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4da763-2940-4cf7-8619-8f40131f2e60 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:56.264732 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02d656-0143-433e-8384-9039bef7583b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:56.265695 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02d656-0143-433e-8384-9039bef7583b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:56.266121 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02d656-0143-433e-8384-9039bef7583b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:56.266586 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db02d656-0143-433e-8384-9039bef7583b admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:56.267532 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db02d656-0143-433e-8384-9039bef7583b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:33:56.268021 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db02d656-0143-433e-8384-9039bef7583b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:56.273715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db02d656-0143-433e-8384-9039bef7583b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:56.310089 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 945/1899] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:33:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:33:56.311150 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02d656-0143-433e-8384-9039bef7583b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:56.333395 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db02d656-0143-433e-8384-9039bef7583b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:56.347500 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3040d-25a9-49d2-9e31-438e4de13241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:56.348362 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3040d-25a9-49d2-9e31-438e4de13241 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:56.348648 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3040d-25a9-49d2-9e31-438e4de13241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:56.348935 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3040d-25a9-49d2-9e31-438e4de13241 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:56.349647 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3040d-25a9-49d2-9e31-438e4de13241 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:56.351611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3040d-25a9-49d2-9e31-438e4de13241 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:56.354737 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9f2970-d947-464d-b453-f15c5920b609 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:56.355599 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9f2970-d947-464d-b453-f15c5920b609 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:56.371265 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 955/1900] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 12:33:56.421581 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 946/1901] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:56 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:56.426703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f2970-d947-464d-b453-f15c5920b609 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:56.446654 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f2970-d947-464d-b453-f15c5920b609 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:56.591791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:56.633813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0324821-aa7c-4e00-9639-774b25e676b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:56.634619 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0324821-aa7c-4e00-9639-774b25e676b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:56.634945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0324821-aa7c-4e00-9639-774b25e676b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:56.635228 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0324821-aa7c-4e00-9639-774b25e676b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:56.635933 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0324821-aa7c-4e00-9639-774b25e676b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:56.638260 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0324821-aa7c-4e00-9639-774b25e676b5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:56.675874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 956/1902] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:56 2022] POST /identity/v3/users => generated 290 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:33:56.679425 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3040d-25a9-49d2-9e31-438e4de13241 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:56.695165 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3040d-25a9-49d2-9e31-438e4de13241 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:56.709161 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bfbc3-4653-4377-8822-6de6f01ae7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:56.709980 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bfbc3-4653-4377-8822-6de6f01ae7fe admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:56.710382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bfbc3-4653-4377-8822-6de6f01ae7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:56.710820 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bfbc3-4653-4377-8822-6de6f01ae7fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:56.712754 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10bfbc3-4653-4377-8822-6de6f01ae7fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:56.717690 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10bfbc3-4653-4377-8822-6de6f01ae7fe admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:56.746313 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 947/1903] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:56.750265 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0324821-aa7c-4e00-9639-774b25e676b5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:56.774948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0324821-aa7c-4e00-9639-774b25e676b5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:56.796465 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc2f1d5-a98d-4904-98c5-ad41b088f36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:56.797393 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2f1d5-a98d-4904-98c5-ad41b088f36b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:56.797783 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2f1d5-a98d-4904-98c5-ad41b088f36b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:56.798182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc2f1d5-a98d-4904-98c5-ad41b088f36b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:56.799491 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc2f1d5-a98d-4904-98c5-ad41b088f36b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:56.802627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc2f1d5-a98d-4904-98c5-ad41b088f36b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:56.821422 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 957/1904] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:56.825225 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bfbc3-4653-4377-8822-6de6f01ae7fe admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:56.841239 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bfbc3-4653-4377-8822-6de6f01ae7fe admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:56.849160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5fdc6-3fe7-454c-80f8-b0e981324363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:56.850573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5fdc6-3fe7-454c-80f8-b0e981324363 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:56.851341 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5fdc6-3fe7-454c-80f8-b0e981324363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:56.851606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5fdc6-3fe7-454c-80f8-b0e981324363 admin admin] PATH_INFO: `/v3/projects/8f33c510a8d94549bd60e20c33e2b3d2/users/caf1ccad49e449c3a28e972463579ca1/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:56.875964 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5fdc6-3fe7-454c-80f8-b0e981324363 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f33c510a8d94549bd60e20c33e2b3d2, user_id=caf1ccad49e449c3a28e972463579ca1, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:56.881486 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d5fdc6-3fe7-454c-80f8-b0e981324363 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:56.901802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 948/1905] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:56 2022] PUT /identity/v3/projects/8f33c510a8d94549bd60e20c33e2b3d2/users/caf1ccad49e449c3a28e972463579ca1/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:56.913406 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8da590-3792-4f05-a09a-541ebf454154 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:56.913851 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8da590-3792-4f05-a09a-541ebf454154 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:56.914271 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8da590-3792-4f05-a09a-541ebf454154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:56.967683 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2b8da590-3792-4f05-a09a-541ebf454154 None None] MFA Rules not processed for user `caf1ccad49e449c3a28e972463579ca1`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:56.969259 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2b8da590-3792-4f05-a09a-541ebf454154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:57.085488 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 958/1906] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:57.089844 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5fdc6-3fe7-454c-80f8-b0e981324363 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:57.179843 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5fdc6-3fe7-454c-80f8-b0e981324363 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:57.186493 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff93e21-6ef7-46c2-b08c-ccd76b882d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:57.187275 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff93e21-6ef7-46c2-b08c-ccd76b882d2d admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:57.187657 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff93e21-6ef7-46c2-b08c-ccd76b882d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:57.188160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff93e21-6ef7-46c2-b08c-ccd76b882d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:57.189073 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff93e21-6ef7-46c2-b08c-ccd76b882d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:57.191635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff93e21-6ef7-46c2-b08c-ccd76b882d2d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:57.230069 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 949/1907] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:33:57 2022] POST /identity/v3/projects => generated 381 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:57.234256 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8da590-3792-4f05-a09a-541ebf454154 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:57.248552 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8da590-3792-4f05-a09a-541ebf454154 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:57.366389 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32f171-4537-4011-9d3e-fa4be0cef724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:57.366638 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32f171-4537-4011-9d3e-fa4be0cef724 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:57.366926 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32f171-4537-4011-9d3e-fa4be0cef724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:57.367206 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32f171-4537-4011-9d3e-fa4be0cef724 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:57.371114 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd32f171-4537-4011-9d3e-fa4be0cef724 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:57.374965 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd32f171-4537-4011-9d3e-fa4be0cef724 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:57.424855 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 959/1908] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:33:57 2022] POST /identity/v3/users => generated 297 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:33:57.428736 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff93e21-6ef7-46c2-b08c-ccd76b882d2d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:57.445369 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff93e21-6ef7-46c2-b08c-ccd76b882d2d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:57.455588 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84e576-047c-4c16-a702-10e95a30667b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:57.458161 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84e576-047c-4c16-a702-10e95a30667b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:57.458859 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84e576-047c-4c16-a702-10e95a30667b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:57.460556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84e576-047c-4c16-a702-10e95a30667b admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:57.462017 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84e576-047c-4c16-a702-10e95a30667b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:57.465626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84e576-047c-4c16-a702-10e95a30667b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:57.483345 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 950/1909] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:57.489120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32f171-4537-4011-9d3e-fa4be0cef724 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:57.508006 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32f171-4537-4011-9d3e-fa4be0cef724 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:57.515320 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858915e9-6d46-4fa7-863a-12ac9ae5ef27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:57.518119 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858915e9-6d46-4fa7-863a-12ac9ae5ef27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:57.518803 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858915e9-6d46-4fa7-863a-12ac9ae5ef27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:57.520022 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858915e9-6d46-4fa7-863a-12ac9ae5ef27 admin admin] PATH_INFO: `/v3/projects/5db63b28490b43cc9614ce65395096d8/users/6d33c8dc0db94786a3a85df092eeb136/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:57.565079 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858915e9-6d46-4fa7-863a-12ac9ae5ef27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5db63b28490b43cc9614ce65395096d8, user_id=6d33c8dc0db94786a3a85df092eeb136, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:57.572210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858915e9-6d46-4fa7-863a-12ac9ae5ef27 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:57.606003 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 960/1910] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:57 2022] PUT /identity/v3/projects/5db63b28490b43cc9614ce65395096d8/users/6d33c8dc0db94786a3a85df092eeb136/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:57.609777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84e576-047c-4c16-a702-10e95a30667b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:57.624636 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84e576-047c-4c16-a702-10e95a30667b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:57.689061 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:57.707294 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5ff17-0031-4521-bd13-c894ef3771c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:57.708532 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5ff17-0031-4521-bd13-c894ef3771c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:57.709373 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5ff17-0031-4521-bd13-c894ef3771c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:57.710010 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5ff17-0031-4521-bd13-c894ef3771c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:57.715514 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de5ff17-0031-4521-bd13-c894ef3771c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:57.718994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de5ff17-0031-4521-bd13-c894ef3771c8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:57.781153 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 951/1911] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:33:57 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:33:57.785695 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858915e9-6d46-4fa7-863a-12ac9ae5ef27 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:57.803139 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858915e9-6d46-4fa7-863a-12ac9ae5ef27 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:57.814024 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1fa63-1b2a-44f5-902f-899e09033a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:57.820091 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1fa63-1b2a-44f5-902f-899e09033a3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:57.821098 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1fa63-1b2a-44f5-902f-899e09033a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:57.822318 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1fa63-1b2a-44f5-902f-899e09033a3f admin admin] PATH_INFO: `/v3/projects/5db63b28490b43cc9614ce65395096d8/users/6d33c8dc0db94786a3a85df092eeb136/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:57.828526 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1fa63-1b2a-44f5-902f-899e09033a3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5db63b28490b43cc9614ce65395096d8, user_id=6d33c8dc0db94786a3a85df092eeb136, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:57.849070 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1fa63-1b2a-44f5-902f-899e09033a3f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:57.889680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 961/1912] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:33:57 2022] PUT /identity/v3/projects/5db63b28490b43cc9614ce65395096d8/users/6d33c8dc0db94786a3a85df092eeb136/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:57.905315 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218035e9-02e8-4e42-a280-a7a3aadbdf36 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:57.906154 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218035e9-02e8-4e42-a280-a7a3aadbdf36 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:57.907032 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218035e9-02e8-4e42-a280-a7a3aadbdf36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:57.973811 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-218035e9-02e8-4e42-a280-a7a3aadbdf36 None None] MFA Rules not processed for user `6d33c8dc0db94786a3a85df092eeb136`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:57.975395 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-218035e9-02e8-4e42-a280-a7a3aadbdf36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:58.092336 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 952/1913] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:58.104848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1fa63-1b2a-44f5-902f-899e09033a3f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:58.107035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f2d1fa63-1b2a-44f5-902f-899e09033a3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:58.118533 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:58.246742 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d83e9-4fcf-4167-840b-4c15c06660b5 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:58.247137 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d83e9-4fcf-4167-840b-4c15c06660b5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:58.247431 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d83e9-4fcf-4167-840b-4c15c06660b5 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:58.249837 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 953/1914] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jun 29 12:33:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:33:58.255456 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1fa63-1b2a-44f5-902f-899e09033a3f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:58.263381 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f9696-5d7f-492c-ae42-e7dcb23ede64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:58.279992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f9696-5d7f-492c-ae42-e7dcb23ede64 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:58.280168 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f9696-5d7f-492c-ae42-e7dcb23ede64 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:58.280643 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f9696-5d7f-492c-ae42-e7dcb23ede64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:58.296172 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f9696-5d7f-492c-ae42-e7dcb23ede64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:58.301002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f9696-5d7f-492c-ae42-e7dcb23ede64 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:58.342651 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a672025-6d7d-453e-ae2b-92f6a41b1cad None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:58.342651 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a672025-6d7d-453e-ae2b-92f6a41b1cad None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:58.342651 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a672025-6d7d-453e-ae2b-92f6a41b1cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:58.342874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 962/1915] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:58 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:58.348645 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eecec8-3153-4635-bfdd-99e941ec778b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:58.348973 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eecec8-3153-4635-bfdd-99e941ec778b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:58.370779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eecec8-3153-4635-bfdd-99e941ec778b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:58.408802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3a672025-6d7d-453e-ae2b-92f6a41b1cad None None] MFA Rules not processed for user `3ac7391a3c41499d94e5f97478797313`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:58.410883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3a672025-6d7d-453e-ae2b-92f6a41b1cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:58.431642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-31eecec8-3153-4635-bfdd-99e941ec778b None None] MFA Rules not processed for user `bba6a17fc19645c29fce6fa655095a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:58.432899 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-31eecec8-3153-4635-bfdd-99e941ec778b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:58.600723 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 963/1916] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:58.652849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 954/1917] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jun 29 12:33:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:58.842683 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eecec8-3153-4635-bfdd-99e941ec778b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:58.844160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-31eecec8-3153-4635-bfdd-99e941ec778b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:58.892322 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eecec8-3153-4635-bfdd-99e941ec778b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:58.899164 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf7c6ab-ab00-4da3-93fa-06d440f40794 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:58.899845 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7c6ab-ab00-4da3-93fa-06d440f40794 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:58.900128 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7c6ab-ab00-4da3-93fa-06d440f40794 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:58.900568 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7c6ab-ab00-4da3-93fa-06d440f40794 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:58.914444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7c6ab-ab00-4da3-93fa-06d440f40794 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:58.917507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf7c6ab-ab00-4da3-93fa-06d440f40794 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:58.935879 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 964/1918] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:59.264271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a672025-6d7d-453e-ae2b-92f6a41b1cad None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:59.371693 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a672025-6d7d-453e-ae2b-92f6a41b1cad None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:59.384739 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080a43f-709e-43a9-9cb3-b185e8808f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:59.386273 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080a43f-709e-43a9-9cb3-b185e8808f11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:59.386606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080a43f-709e-43a9-9cb3-b185e8808f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:59.387055 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8080a43f-709e-43a9-9cb3-b185e8808f11 admin admin] PATH_INFO: `/v3/users/caf1ccad49e449c3a28e972463579ca1` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:59.388385 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080a43f-709e-43a9-9cb3-b185e8808f11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=caf1ccad49e449c3a28e972463579ca1)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:59.390372 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8080a43f-709e-43a9-9cb3-b185e8808f11 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:59.396768 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:33:59.480277 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8080a43f-709e-43a9-9cb3-b185e8808f11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caf1ccad49e449c3a28e972463579ca1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:59.494356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8080a43f-709e-43a9-9cb3-b185e8808f11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caf1ccad49e449c3a28e972463579ca1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:59.518662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8080a43f-709e-43a9-9cb3-b185e8808f11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caf1ccad49e449c3a28e972463579ca1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:59.519951 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8080a43f-709e-43a9-9cb3-b185e8808f11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caf1ccad49e449c3a28e972463579ca1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:59.521282 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8080a43f-709e-43a9-9cb3-b185e8808f11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caf1ccad49e449c3a28e972463579ca1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:33:59.537802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 955/1919] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:33:59 2022] DELETE /identity/v3/users/caf1ccad49e449c3a28e972463579ca1 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:33:59.539321 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b199f83-c005-4662-b9df-6a05b1f414c5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:59.539616 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b199f83-c005-4662-b9df-6a05b1f414c5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:59.539900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b199f83-c005-4662-b9df-6a05b1f414c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:59.572193 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8b199f83-c005-4662-b9df-6a05b1f414c5 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:33:59.573661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8b199f83-c005-4662-b9df-6a05b1f414c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:59.674916 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 965/1920] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:33:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:33:59.683440 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080a43f-709e-43a9-9cb3-b185e8808f11 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:59.685113 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8080a43f-709e-43a9-9cb3-b185e8808f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:59.759562 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8080a43f-709e-43a9-9cb3-b185e8808f11 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:59.763393 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b6afe0-fd00-47eb-87bc-942d5c5a897d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:59.764176 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6afe0-fd00-47eb-87bc-942d5c5a897d service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:59.764561 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6afe0-fd00-47eb-87bc-942d5c5a897d service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:59.764807 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6afe0-fd00-47eb-87bc-942d5c5a897d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:59.775227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b6afe0-fd00-47eb-87bc-942d5c5a897d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:59.776657 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b6afe0-fd00-47eb-87bc-942d5c5a897d service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:33:59.790374 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 956/1921] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:33:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:33:59.918851 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b199f83-c005-4662-b9df-6a05b1f414c5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:33:59.920501 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8b199f83-c005-4662-b9df-6a05b1f414c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:33:59.944243 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b199f83-c005-4662-b9df-6a05b1f414c5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:33:59.962144 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e5af9-be77-46ef-85cb-3d57fb2d31c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:33:59.963864 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e5af9-be77-46ef-85cb-3d57fb2d31c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:33:59.964326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e5af9-be77-46ef-85cb-3d57fb2d31c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:33:59.964876 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e5af9-be77-46ef-85cb-3d57fb2d31c2 admin admin] PATH_INFO: `/v3/users/f339584f5ea64315a3a664b9352552cc` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:33:59.967667 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e5af9-be77-46ef-85cb-3d57fb2d31c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f339584f5ea64315a3a664b9352552cc)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:33:59.971680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e5af9-be77-46ef-85cb-3d57fb2d31c2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:00.049195 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:00.073268 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c47e5af9-be77-46ef-85cb-3d57fb2d31c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f339584f5ea64315a3a664b9352552cc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:00.074165 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c47e5af9-be77-46ef-85cb-3d57fb2d31c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f339584f5ea64315a3a664b9352552cc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:00.091048 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b6afe0-fd00-47eb-87bc-942d5c5a897d service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:00.091182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c47e5af9-be77-46ef-85cb-3d57fb2d31c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f339584f5ea64315a3a664b9352552cc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:00.091769 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c47e5af9-be77-46ef-85cb-3d57fb2d31c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f339584f5ea64315a3a664b9352552cc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:00.093049 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a6b6afe0-fd00-47eb-87bc-942d5c5a897d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:00.100057 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c47e5af9-be77-46ef-85cb-3d57fb2d31c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f339584f5ea64315a3a664b9352552cc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:00.111697 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b6afe0-fd00-47eb-87bc-942d5c5a897d service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:00.122217 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 966/1922] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:33:59 2022] DELETE /identity/v3/users/f339584f5ea64315a3a664b9352552cc => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:00.123978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf93e9d-c47d-4658-b49e-fb7e84df1e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:00.126702 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf93e9d-c47d-4658-b49e-fb7e84df1e33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:00.127988 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf93e9d-c47d-4658-b49e-fb7e84df1e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:00.128611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf93e9d-c47d-4658-b49e-fb7e84df1e33 admin admin] PATH_INFO: `/v3/users/7585ec5b962746fd98f07d5b45b6be96` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:00.130802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf93e9d-c47d-4658-b49e-fb7e84df1e33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7585ec5b962746fd98f07d5b45b6be96)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:00.132771 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf257a6d-a03c-446b-b001-a2308ebfb795 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:00.133072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf257a6d-a03c-446b-b001-a2308ebfb795 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:00.133384 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf257a6d-a03c-446b-b001-a2308ebfb795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:00.153047 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf93e9d-c47d-4658-b49e-fb7e84df1e33 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:00.196437 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bf257a6d-a03c-446b-b001-a2308ebfb795 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:00.198490 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bf257a6d-a03c-446b-b001-a2308ebfb795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:00.241570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7cf93e9d-c47d-4658-b49e-fb7e84df1e33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7585ec5b962746fd98f07d5b45b6be96'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:00.324888 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 967/1923] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:00.357719 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7cf93e9d-c47d-4658-b49e-fb7e84df1e33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7585ec5b962746fd98f07d5b45b6be96'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:00.365020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf257a6d-a03c-446b-b001-a2308ebfb795 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:00.367576 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bf257a6d-a03c-446b-b001-a2308ebfb795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:00.389924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7cf93e9d-c47d-4658-b49e-fb7e84df1e33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7585ec5b962746fd98f07d5b45b6be96'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:00.391199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7cf93e9d-c47d-4658-b49e-fb7e84df1e33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7585ec5b962746fd98f07d5b45b6be96'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:00.392037 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7cf93e9d-c47d-4658-b49e-fb7e84df1e33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7585ec5b962746fd98f07d5b45b6be96'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:00.401769 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:00.415405 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 957/1924] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:34:00 2022] DELETE /identity/v3/users/7585ec5b962746fd98f07d5b45b6be96 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:00.417003 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf93e9d-c47d-4658-b49e-fb7e84df1e33 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:00.420827 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7cf93e9d-c47d-4658-b49e-fb7e84df1e33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:00.497969 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf257a6d-a03c-446b-b001-a2308ebfb795 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:00.508175 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf93e9d-c47d-4658-b49e-fb7e84df1e33 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:00.515148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9b1de-be7d-4532-bd23-16d290932847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:00.516324 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9b1de-be7d-4532-bd23-16d290932847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:00.516597 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9b1de-be7d-4532-bd23-16d290932847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:00.516881 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9b1de-be7d-4532-bd23-16d290932847 admin admin] PATH_INFO: `/v3/projects/8f33c510a8d94549bd60e20c33e2b3d2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:00.520791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9b1de-be7d-4532-bd23-16d290932847 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f33c510a8d94549bd60e20c33e2b3d2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:00.522543 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9b1de-be7d-4532-bd23-16d290932847 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:00.595266 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2fe9b1de-be7d-4532-bd23-16d290932847 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f33c510a8d94549bd60e20c33e2b3d2', 'request_id': 'req-2fe9b1de-be7d-4532-bd23-16d290932847'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:00.600497 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7979a90-2285-4574-b5a9-3fd9f151b18c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:00.601585 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7979a90-2285-4574-b5a9-3fd9f151b18c service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:00.601997 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7979a90-2285-4574-b5a9-3fd9f151b18c service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:00.602453 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7979a90-2285-4574-b5a9-3fd9f151b18c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:00.604278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2fe9b1de-be7d-4532-bd23-16d290932847 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f33c510a8d94549bd60e20c33e2b3d2', 'request_id': 'req-2fe9b1de-be7d-4532-bd23-16d290932847'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:00.604701 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d7979a90-2285-4574-b5a9-3fd9f151b18c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:00.612408 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2fe9b1de-be7d-4532-bd23-16d290932847 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f33c510a8d94549bd60e20c33e2b3d2', 'request_id': 'req-2fe9b1de-be7d-4532-bd23-16d290932847'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:00.621701 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7979a90-2285-4574-b5a9-3fd9f151b18c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:00.623598 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7979a90-2285-4574-b5a9-3fd9f151b18c service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:00.632865 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2fe9b1de-be7d-4532-bd23-16d290932847 admin admin] The token cache is being invalidate because project 8f33c510a8d94549bd60e20c33e2b3d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:00.633171 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2fe9b1de-be7d-4532-bd23-16d290932847 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:00.634060 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2fe9b1de-be7d-4532-bd23-16d290932847 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:00.636891 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 958/1925] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:00 2022] DELETE /identity/v3/projects/8f33c510a8d94549bd60e20c33e2b3d2 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:00.640020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064939a-1b8d-429e-a848-baad24ca17d7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:00.640020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064939a-1b8d-429e-a848-baad24ca17d7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:00.640175 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064939a-1b8d-429e-a848-baad24ca17d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:00.648126 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 968/1926] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:00.650273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7979a90-2285-4574-b5a9-3fd9f151b18c service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:00.661598 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7979a90-2285-4574-b5a9-3fd9f151b18c service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:00.662803 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2064939a-1b8d-429e-a848-baad24ca17d7 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:00.664187 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2064939a-1b8d-429e-a848-baad24ca17d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:00.668719 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:00.779613 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2284693f-e72c-4df9-b7f0-f42d667b5584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:00.780553 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284693f-e72c-4df9-b7f0-f42d667b5584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:00.780843 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284693f-e72c-4df9-b7f0-f42d667b5584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:00.781121 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284693f-e72c-4df9-b7f0-f42d667b5584 admin admin] PATH_INFO: `/v3/users/6d33c8dc0db94786a3a85df092eeb136` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:00.782377 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2284693f-e72c-4df9-b7f0-f42d667b5584 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d33c8dc0db94786a3a85df092eeb136)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:00.784333 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2284693f-e72c-4df9-b7f0-f42d667b5584 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:00.792840 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 959/1927] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:00.828856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2064939a-1b8d-429e-a848-baad24ca17d7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:00.832310 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2064939a-1b8d-429e-a848-baad24ca17d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:00.853747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2284693f-e72c-4df9-b7f0-f42d667b5584 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d33c8dc0db94786a3a85df092eeb136'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:00.858341 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2284693f-e72c-4df9-b7f0-f42d667b5584 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d33c8dc0db94786a3a85df092eeb136'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:00.887381 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2284693f-e72c-4df9-b7f0-f42d667b5584 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d33c8dc0db94786a3a85df092eeb136'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:00.888196 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2284693f-e72c-4df9-b7f0-f42d667b5584 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d33c8dc0db94786a3a85df092eeb136'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:00.891326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:00.902657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2284693f-e72c-4df9-b7f0-f42d667b5584 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d33c8dc0db94786a3a85df092eeb136'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:00.928085 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2064939a-1b8d-429e-a848-baad24ca17d7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:00.932087 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e0779-9d48-48b6-9f85-49a4437ac6e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:00.933171 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e0779-9d48-48b6-9f85-49a4437ac6e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:00.933292 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e0779-9d48-48b6-9f85-49a4437ac6e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:00.933492 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e0779-9d48-48b6-9f85-49a4437ac6e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:00.934294 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 969/1928] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:34:00 2022] DELETE /identity/v3/users/6d33c8dc0db94786a3a85df092eeb136 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:00.935840 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f73e0779-9d48-48b6-9f85-49a4437ac6e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:00.952642 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e0779-9d48-48b6-9f85-49a4437ac6e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:00.955423 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e0779-9d48-48b6-9f85-49a4437ac6e8 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:00.969318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 960/1929] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:01.384148 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2284693f-e72c-4df9-b7f0-f42d667b5584 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:01.385928 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2284693f-e72c-4df9-b7f0-f42d667b5584 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:01.521357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2284693f-e72c-4df9-b7f0-f42d667b5584 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:01.526502 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78e75f-eeb5-4cd4-95f7-e60a9c4e9529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:01.527558 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78e75f-eeb5-4cd4-95f7-e60a9c4e9529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:01.527976 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78e75f-eeb5-4cd4-95f7-e60a9c4e9529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:01.528321 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78e75f-eeb5-4cd4-95f7-e60a9c4e9529 admin admin] PATH_INFO: `/v3/projects/f3394a1d2a42406b89c7b100c4809585` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:01.530247 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a78e75f-eeb5-4cd4-95f7-e60a9c4e9529 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3394a1d2a42406b89c7b100c4809585)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:01.532595 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a78e75f-eeb5-4cd4-95f7-e60a9c4e9529 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:01.576002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:01.595365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5a78e75f-eeb5-4cd4-95f7-e60a9c4e9529 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3394a1d2a42406b89c7b100c4809585', 'request_id': 'req-5a78e75f-eeb5-4cd4-95f7-e60a9c4e9529'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:01.605445 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5a78e75f-eeb5-4cd4-95f7-e60a9c4e9529 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3394a1d2a42406b89c7b100c4809585', 'request_id': 'req-5a78e75f-eeb5-4cd4-95f7-e60a9c4e9529'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:01.618384 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5a78e75f-eeb5-4cd4-95f7-e60a9c4e9529 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3394a1d2a42406b89c7b100c4809585', 'request_id': 'req-5a78e75f-eeb5-4cd4-95f7-e60a9c4e9529'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:01.631885 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5a78e75f-eeb5-4cd4-95f7-e60a9c4e9529 admin admin] The token cache is being invalidate because project f3394a1d2a42406b89c7b100c4809585 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:01.632381 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5a78e75f-eeb5-4cd4-95f7-e60a9c4e9529 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:01.633040 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5a78e75f-eeb5-4cd4-95f7-e60a9c4e9529 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:01.634684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 970/1930] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:01 2022] DELETE /identity/v3/projects/f3394a1d2a42406b89c7b100c4809585 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:01.638612 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e0779-9d48-48b6-9f85-49a4437ac6e8 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:01.649655 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e0779-9d48-48b6-9f85-49a4437ac6e8 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:01.718936 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760f16b-9a56-406e-bb82-3d7c8daae5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:01.720185 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760f16b-9a56-406e-bb82-3d7c8daae5d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:01.720741 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760f16b-9a56-406e-bb82-3d7c8daae5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:01.721738 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760f16b-9a56-406e-bb82-3d7c8daae5d1 admin admin] PATH_INFO: `/v3/users/96af918c89e74cc2947711dec19e4b4f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:01.723986 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4760f16b-9a56-406e-bb82-3d7c8daae5d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96af918c89e74cc2947711dec19e4b4f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:01.726280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4760f16b-9a56-406e-bb82-3d7c8daae5d1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:01.814717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4760f16b-9a56-406e-bb82-3d7c8daae5d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96af918c89e74cc2947711dec19e4b4f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:01.825789 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4760f16b-9a56-406e-bb82-3d7c8daae5d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96af918c89e74cc2947711dec19e4b4f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:01.846761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4760f16b-9a56-406e-bb82-3d7c8daae5d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96af918c89e74cc2947711dec19e4b4f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:01.848920 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4760f16b-9a56-406e-bb82-3d7c8daae5d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96af918c89e74cc2947711dec19e4b4f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:01.849948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4760f16b-9a56-406e-bb82-3d7c8daae5d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96af918c89e74cc2947711dec19e4b4f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:01.854240 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78e75f-eeb5-4cd4-95f7-e60a9c4e9529 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:01.856791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5a78e75f-eeb5-4cd4-95f7-e60a9c4e9529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:01.870478 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 961/1931] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:34:01 2022] DELETE /identity/v3/users/96af918c89e74cc2947711dec19e4b4f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:01.885049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78e75f-eeb5-4cd4-95f7-e60a9c4e9529 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:01.991062 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e296668-bc54-4c05-a6e8-b37c43572ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:01.992019 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e296668-bc54-4c05-a6e8-b37c43572ad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:01.992296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e296668-bc54-4c05-a6e8-b37c43572ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:01.992624 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e296668-bc54-4c05-a6e8-b37c43572ad0 admin admin] PATH_INFO: `/v3/projects/31d98c906480468b830fee7c37d725a4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:01.994443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e296668-bc54-4c05-a6e8-b37c43572ad0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31d98c906480468b830fee7c37d725a4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:01.996776 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e296668-bc54-4c05-a6e8-b37c43572ad0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:02.010660 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760f16b-9a56-406e-bb82-3d7c8daae5d1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:02.012660 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4760f16b-9a56-406e-bb82-3d7c8daae5d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:02.041004 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760f16b-9a56-406e-bb82-3d7c8daae5d1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:02.048253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2deb4-7ad8-4206-b158-d0f551003a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:02.049125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2deb4-7ad8-4206-b158-d0f551003a9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:02.050176 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2deb4-7ad8-4206-b158-d0f551003a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:02.050491 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2deb4-7ad8-4206-b158-d0f551003a9f admin admin] PATH_INFO: `/v3/projects/5db63b28490b43cc9614ce65395096d8` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:02.051901 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a2deb4-7ad8-4206-b158-d0f551003a9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5db63b28490b43cc9614ce65395096d8)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:02.054179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a2deb4-7ad8-4206-b158-d0f551003a9f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:02.109381 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e296668-bc54-4c05-a6e8-b37c43572ad0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31d98c906480468b830fee7c37d725a4', 'request_id': 'req-2e296668-bc54-4c05-a6e8-b37c43572ad0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:02.134108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e296668-bc54-4c05-a6e8-b37c43572ad0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31d98c906480468b830fee7c37d725a4', 'request_id': 'req-2e296668-bc54-4c05-a6e8-b37c43572ad0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:02.146085 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e296668-bc54-4c05-a6e8-b37c43572ad0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31d98c906480468b830fee7c37d725a4', 'request_id': 'req-2e296668-bc54-4c05-a6e8-b37c43572ad0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:02.170241 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:02.189136 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e296668-bc54-4c05-a6e8-b37c43572ad0 admin admin] The token cache is being invalidate because project 31d98c906480468b830fee7c37d725a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:02.191695 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e296668-bc54-4c05-a6e8-b37c43572ad0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:02.193043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-77a2deb4-7ad8-4206-b158-d0f551003a9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5db63b28490b43cc9614ce65395096d8', 'request_id': 'req-77a2deb4-7ad8-4206-b158-d0f551003a9f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:02.193161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2e296668-bc54-4c05-a6e8-b37c43572ad0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:02.211663 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-77a2deb4-7ad8-4206-b158-d0f551003a9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5db63b28490b43cc9614ce65395096d8', 'request_id': 'req-77a2deb4-7ad8-4206-b158-d0f551003a9f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:02.212035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 971/1932] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:01 2022] DELETE /identity/v3/projects/31d98c906480468b830fee7c37d725a4 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:02.215210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e296668-bc54-4c05-a6e8-b37c43572ad0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:02.217578 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2e296668-bc54-4c05-a6e8-b37c43572ad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:02.232646 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-77a2deb4-7ad8-4206-b158-d0f551003a9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5db63b28490b43cc9614ce65395096d8', 'request_id': 'req-77a2deb4-7ad8-4206-b158-d0f551003a9f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:02.260494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-77a2deb4-7ad8-4206-b158-d0f551003a9f admin admin] The token cache is being invalidate because project 5db63b28490b43cc9614ce65395096d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:02.260805 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-77a2deb4-7ad8-4206-b158-d0f551003a9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:02.261493 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-77a2deb4-7ad8-4206-b158-d0f551003a9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:02.269979 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 962/1933] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:02 2022] DELETE /identity/v3/projects/5db63b28490b43cc9614ce65395096d8 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:34:02.275854 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269d52d-c5c7-4ca0-84d1-a7f33d5c1bf6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:02.276153 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269d52d-c5c7-4ca0-84d1-a7f33d5c1bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:02.276443 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b269d52d-c5c7-4ca0-84d1-a7f33d5c1bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:02.318642 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b269d52d-c5c7-4ca0-84d1-a7f33d5c1bf6 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:02.320410 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b269d52d-c5c7-4ca0-84d1-a7f33d5c1bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:02.354630 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:02.387053 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e296668-bc54-4c05-a6e8-b37c43572ad0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:02.505022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 963/1934] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:02.514723 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df11e0f-3649-4dc2-863e-ba52bb10949b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:02.515038 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df11e0f-3649-4dc2-863e-ba52bb10949b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:02.515625 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df11e0f-3649-4dc2-863e-ba52bb10949b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:02.545572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1df11e0f-3649-4dc2-863e-ba52bb10949b None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:02.547968 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1df11e0f-3649-4dc2-863e-ba52bb10949b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:02.561829 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 964/1935] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:02.562926 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df11e0f-3649-4dc2-863e-ba52bb10949b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:02.574860 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df11e0f-3649-4dc2-863e-ba52bb10949b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:02.580750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d7b515-94c4-4b9d-9197-bb0b1a9bad22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:02.582036 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7b515-94c4-4b9d-9197-bb0b1a9bad22 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:02.582398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7b515-94c4-4b9d-9197-bb0b1a9bad22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:02.582780 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7b515-94c4-4b9d-9197-bb0b1a9bad22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:02.583666 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d7b515-94c4-4b9d-9197-bb0b1a9bad22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:34:02.584042 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d7b515-94c4-4b9d-9197-bb0b1a9bad22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:02.585906 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d7b515-94c4-4b9d-9197-bb0b1a9bad22 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:02.588125 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552bcf62-a5b7-427d-b8b8-1f46c568b96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:02.595263 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552bcf62-a5b7-427d-b8b8-1f46c568b96a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:02.595699 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552bcf62-a5b7-427d-b8b8-1f46c568b96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:02.595946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552bcf62-a5b7-427d-b8b8-1f46c568b96a admin admin] PATH_INFO: `/v3/users/9b16af7a836b4a77844e0b186f264b93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:02.598201 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552bcf62-a5b7-427d-b8b8-1f46c568b96a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b16af7a836b4a77844e0b186f264b93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:02.600873 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552bcf62-a5b7-427d-b8b8-1f46c568b96a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:02.608286 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 965/1936] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:34:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:34:02.609249 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d7b515-94c4-4b9d-9197-bb0b1a9bad22 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:02.621689 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d7b515-94c4-4b9d-9197-bb0b1a9bad22 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:02.627181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a1620-93b4-4de0-bad7-c3ce8fd51900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:02.627930 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a1620-93b4-4de0-bad7-c3ce8fd51900 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:02.629038 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a1620-93b4-4de0-bad7-c3ce8fd51900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:02.629333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40a1620-93b4-4de0-bad7-c3ce8fd51900 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:02.630052 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a1620-93b4-4de0-bad7-c3ce8fd51900 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:02.632091 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40a1620-93b4-4de0-bad7-c3ce8fd51900 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:02.688461 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 966/1937] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:34:02 2022] POST /identity/v3/projects => generated 373 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:02.689424 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a1620-93b4-4de0-bad7-c3ce8fd51900 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:02.712635 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-552bcf62-a5b7-427d-b8b8-1f46c568b96a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b16af7a836b4a77844e0b186f264b93'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:02.714818 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-552bcf62-a5b7-427d-b8b8-1f46c568b96a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b16af7a836b4a77844e0b186f264b93'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:02.745404 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-552bcf62-a5b7-427d-b8b8-1f46c568b96a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b16af7a836b4a77844e0b186f264b93'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:02.746222 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-552bcf62-a5b7-427d-b8b8-1f46c568b96a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b16af7a836b4a77844e0b186f264b93'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:02.759326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-552bcf62-a5b7-427d-b8b8-1f46c568b96a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b16af7a836b4a77844e0b186f264b93'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:02.781877 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 972/1938] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:34:02 2022] DELETE /identity/v3/users/9b16af7a836b4a77844e0b186f264b93 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:02.813525 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40a1620-93b4-4de0-bad7-c3ce8fd51900 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:02.901530 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:02.954764 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f75f21-3067-4e62-bba6-2fc6cc6fa266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:02.956045 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f75f21-3067-4e62-bba6-2fc6cc6fa266 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:02.960685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f75f21-3067-4e62-bba6-2fc6cc6fa266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:02.960685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f75f21-3067-4e62-bba6-2fc6cc6fa266 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:02.960685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f75f21-3067-4e62-bba6-2fc6cc6fa266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:02.965305 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f75f21-3067-4e62-bba6-2fc6cc6fa266 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:03.026937 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 967/1939] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:34:02 2022] POST /identity/v3/users => generated 287 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:03.031093 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552bcf62-a5b7-427d-b8b8-1f46c568b96a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:03.033772 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-552bcf62-a5b7-427d-b8b8-1f46c568b96a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:03.061767 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552bcf62-a5b7-427d-b8b8-1f46c568b96a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:03.082221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090d120-47d3-4638-9552-a05bab08731b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:03.084587 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090d120-47d3-4638-9552-a05bab08731b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:03.084960 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090d120-47d3-4638-9552-a05bab08731b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:03.085308 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090d120-47d3-4638-9552-a05bab08731b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:03.086182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090d120-47d3-4638-9552-a05bab08731b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:03.089191 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5090d120-47d3-4638-9552-a05bab08731b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:03.119730 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f75f21-3067-4e62-bba6-2fc6cc6fa266 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:03.121070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-63f75f21-3067-4e62-bba6-2fc6cc6fa266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:03.130345 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 973/1940] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:03.133895 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090d120-47d3-4638-9552-a05bab08731b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:03.149680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f75f21-3067-4e62-bba6-2fc6cc6fa266 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:03.171756 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4a72a-8c50-4a00-8f9a-526900692df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:03.172651 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4a72a-8c50-4a00-8f9a-526900692df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:03.173172 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090d120-47d3-4638-9552-a05bab08731b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:03.189273 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4a72a-8c50-4a00-8f9a-526900692df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:03.189644 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4a72a-8c50-4a00-8f9a-526900692df7 admin admin] PATH_INFO: `/v3/projects/553daa8dbd9f46c68105ec7830f2ee0b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:03.194724 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4a72a-8c50-4a00-8f9a-526900692df7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=553daa8dbd9f46c68105ec7830f2ee0b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:03.196857 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4a72a-8c50-4a00-8f9a-526900692df7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:03.202171 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e83349d-cd42-4bf1-870a-fd95f0397253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:03.208371 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e83349d-cd42-4bf1-870a-fd95f0397253 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:03.210072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e83349d-cd42-4bf1-870a-fd95f0397253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:03.210371 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e83349d-cd42-4bf1-870a-fd95f0397253 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:03.212473 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e83349d-cd42-4bf1-870a-fd95f0397253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:03.229443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e83349d-cd42-4bf1-870a-fd95f0397253 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:03.247366 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 974/1941] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:03.251626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e83349d-cd42-4bf1-870a-fd95f0397253 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:03.265059 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e83349d-cd42-4bf1-870a-fd95f0397253 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:03.272367 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8010f5-7d82-451f-b44d-d2d92be503d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:03.273985 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8010f5-7d82-451f-b44d-d2d92be503d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:03.274343 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8010f5-7d82-451f-b44d-d2d92be503d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:03.274733 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8010f5-7d82-451f-b44d-d2d92be503d4 admin admin] PATH_INFO: `/v3/projects/91caf8abad6c46039501fd01890434d2/users/743d985dd40d4d35a3c1364ba6622524/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:03.299152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8010f5-7d82-451f-b44d-d2d92be503d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91caf8abad6c46039501fd01890434d2, user_id=743d985dd40d4d35a3c1364ba6622524, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:03.304095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8010f5-7d82-451f-b44d-d2d92be503d4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:03.322899 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 975/1942] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:03 2022] PUT /identity/v3/projects/91caf8abad6c46039501fd01890434d2/users/743d985dd40d4d35a3c1364ba6622524/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:03.335631 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0ce59-cd90-479b-aeb8-a85d80acf519 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:03.335931 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0ce59-cd90-479b-aeb8-a85d80acf519 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:03.336965 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0ce59-cd90-479b-aeb8-a85d80acf519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:03.350991 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f5f4a72a-8c50-4a00-8f9a-526900692df7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '553daa8dbd9f46c68105ec7830f2ee0b', 'request_id': 'req-f5f4a72a-8c50-4a00-8f9a-526900692df7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:03.367136 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f5f4a72a-8c50-4a00-8f9a-526900692df7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '553daa8dbd9f46c68105ec7830f2ee0b', 'request_id': 'req-f5f4a72a-8c50-4a00-8f9a-526900692df7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:03.380513 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f5f4a72a-8c50-4a00-8f9a-526900692df7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '553daa8dbd9f46c68105ec7830f2ee0b', 'request_id': 'req-f5f4a72a-8c50-4a00-8f9a-526900692df7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:03.411133 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f5f4a72a-8c50-4a00-8f9a-526900692df7 admin admin] The token cache is being invalidate because project 553daa8dbd9f46c68105ec7830f2ee0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:03.411439 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f5f4a72a-8c50-4a00-8f9a-526900692df7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:03.414645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f5f4a72a-8c50-4a00-8f9a-526900692df7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:03.416904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 968/1943] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:03 2022] DELETE /identity/v3/projects/553daa8dbd9f46c68105ec7830f2ee0b => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:03.423471 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9297a0c-60b0-48c3-9a0a-d49b9472e48d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:03.424851 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9297a0c-60b0-48c3-9a0a-d49b9472e48d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:03.425722 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9297a0c-60b0-48c3-9a0a-d49b9472e48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:03.445905 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3ac0ce59-cd90-479b-aeb8-a85d80acf519 None None] MFA Rules not processed for user `743d985dd40d4d35a3c1364ba6622524`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:03.449655 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3ac0ce59-cd90-479b-aeb8-a85d80acf519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:03.478505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d9297a0c-60b0-48c3-9a0a-d49b9472e48d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:03.479725 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d9297a0c-60b0-48c3-9a0a-d49b9472e48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:03.496880 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:03.582322 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 976/1944] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:03.592425 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0ce59-cd90-479b-aeb8-a85d80acf519 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:03.594067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3ac0ce59-cd90-479b-aeb8-a85d80acf519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:03.630223 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 969/1945] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:03.631928 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a88a28-2994-44a4-8377-fca7024b86f4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:03.632193 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a88a28-2994-44a4-8377-fca7024b86f4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:03.632507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a88a28-2994-44a4-8377-fca7024b86f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:03.666649 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-05a88a28-2994-44a4-8377-fca7024b86f4 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:03.668068 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-05a88a28-2994-44a4-8377-fca7024b86f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:03.682370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 970/1946] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:34:03.686595 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a88a28-2994-44a4-8377-fca7024b86f4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:03.715784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a88a28-2994-44a4-8377-fca7024b86f4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:03.730611 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0ce59-cd90-479b-aeb8-a85d80acf519 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:03.732855 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d860e8b-b891-492d-bf8d-0f6abae2266c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:03.733572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d860e8b-b891-492d-bf8d-0f6abae2266c admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:03.734568 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d860e8b-b891-492d-bf8d-0f6abae2266c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:03.735942 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d860e8b-b891-492d-bf8d-0f6abae2266c admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:03.739567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d860e8b-b891-492d-bf8d-0f6abae2266c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:34:03.739954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d860e8b-b891-492d-bf8d-0f6abae2266c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:03.744601 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d860e8b-b891-492d-bf8d-0f6abae2266c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:03.745660 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f6ee7-87b9-4efc-ba4b-51eaa454acf5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '096234e9f15541c0b187dc8dd527b0fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:03.747211 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f6ee7-87b9-4efc-ba4b-51eaa454acf5 service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:03.747708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f6ee7-87b9-4efc-ba4b-51eaa454acf5 service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:03.748228 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f6ee7-87b9-4efc-ba4b-51eaa454acf5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:03.766607 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 971/1947] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:34:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:34:03.771588 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d860e8b-b891-492d-bf8d-0f6abae2266c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:03.775567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f6ee7-87b9-4efc-ba4b-51eaa454acf5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:03.777445 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f6ee7-87b9-4efc-ba4b-51eaa454acf5 service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:03.798306 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d860e8b-b891-492d-bf8d-0f6abae2266c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:03.808944 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84d5ac-3921-4973-9ff4-6987cbd027a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:03.809686 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 977/1948] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 12:34:03.810376 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84d5ac-3921-4973-9ff4-6987cbd027a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:03.810763 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84d5ac-3921-4973-9ff4-6987cbd027a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:03.811056 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84d5ac-3921-4973-9ff4-6987cbd027a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:03.812014 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f84d5ac-3921-4973-9ff4-6987cbd027a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:03.815364 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f84d5ac-3921-4973-9ff4-6987cbd027a1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:03.886662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 972/1949] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:34:03 2022] POST /identity/v3/projects => generated 369 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:03.890549 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f6ee7-87b9-4efc-ba4b-51eaa454acf5 service glance] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:03.923306 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f6ee7-87b9-4efc-ba4b-51eaa454acf5 service glance] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:04.046353 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17505745-7b10-43a8-90fb-4e0a881ad109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:04.047304 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17505745-7b10-43a8-90fb-4e0a881ad109 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:04.047703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17505745-7b10-43a8-90fb-4e0a881ad109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:04.048047 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17505745-7b10-43a8-90fb-4e0a881ad109 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:04.048819 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17505745-7b10-43a8-90fb-4e0a881ad109 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:04.055298 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17505745-7b10-43a8-90fb-4e0a881ad109 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:04.090943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 978/1950] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:34:03 2022] POST /identity/v3/users => generated 285 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:04.094338 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84d5ac-3921-4973-9ff4-6987cbd027a1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:04.118212 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84d5ac-3921-4973-9ff4-6987cbd027a1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:04.124867 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7829b-e4db-4feb-9060-8fb5877ffa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:04.127495 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7829b-e4db-4feb-9060-8fb5877ffa89 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:04.127495 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7829b-e4db-4feb-9060-8fb5877ffa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:04.127495 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7829b-e4db-4feb-9060-8fb5877ffa89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:04.127495 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7829b-e4db-4feb-9060-8fb5877ffa89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:04.130841 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7829b-e4db-4feb-9060-8fb5877ffa89 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:04.141057 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:04.154878 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 973/1951] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:04.155305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17505745-7b10-43a8-90fb-4e0a881ad109 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:04.171065 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17505745-7b10-43a8-90fb-4e0a881ad109 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:04.187425 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e99183-df03-4d50-b3f4-bb8d1fd85e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:04.187777 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e99183-df03-4d50-b3f4-bb8d1fd85e4f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:04.189803 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e99183-df03-4d50-b3f4-bb8d1fd85e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:04.190173 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e99183-df03-4d50-b3f4-bb8d1fd85e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:04.192701 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e99183-df03-4d50-b3f4-bb8d1fd85e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:04.197305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e99183-df03-4d50-b3f4-bb8d1fd85e4f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:04.223109 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 979/1952] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:04 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:34:04.227677 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7829b-e4db-4feb-9060-8fb5877ffa89 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:04.274542 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7829b-e4db-4feb-9060-8fb5877ffa89 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:04.293994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b8c584-bd84-48b1-b3b7-163e9875daa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:04.296627 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b8c584-bd84-48b1-b3b7-163e9875daa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:04.298774 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b8c584-bd84-48b1-b3b7-163e9875daa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:04.299084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b8c584-bd84-48b1-b3b7-163e9875daa1 admin admin] PATH_INFO: `/v3/projects/329fe57379ac4f89a92e0394d3a99342/users/6f1324e1021a473f8f0044b2ca28234c/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:04.342970 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b8c584-bd84-48b1-b3b7-163e9875daa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=329fe57379ac4f89a92e0394d3a99342, user_id=6f1324e1021a473f8f0044b2ca28234c, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:04.350577 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b8c584-bd84-48b1-b3b7-163e9875daa1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:04.412336 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 974/1953] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:04 2022] PUT /identity/v3/projects/329fe57379ac4f89a92e0394d3a99342/users/6f1324e1021a473f8f0044b2ca28234c/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:04.415774 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1eae35-27c8-4d79-a182-bbaec66fcac8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:04.416080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1eae35-27c8-4d79-a182-bbaec66fcac8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:04.416363 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1eae35-27c8-4d79-a182-bbaec66fcac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:04.475225 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5a1eae35-27c8-4d79-a182-bbaec66fcac8 None None] MFA Rules not processed for user `6f1324e1021a473f8f0044b2ca28234c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:04.476833 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5a1eae35-27c8-4d79-a182-bbaec66fcac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:04.558003 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:04.568026 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b8c584-bd84-48b1-b3b7-163e9875daa1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:04.571048 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-45b8c584-bd84-48b1-b3b7-163e9875daa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:04.661457 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 980/1954] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:04 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:04.662574 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1eae35-27c8-4d79-a182-bbaec66fcac8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:04.791708 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b8c584-bd84-48b1-b3b7-163e9875daa1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:04.797931 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71cc0eb-eb39-465b-b1b2-50bf45bd2854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:04.799057 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71cc0eb-eb39-465b-b1b2-50bf45bd2854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:04.799507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71cc0eb-eb39-465b-b1b2-50bf45bd2854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:04.799857 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71cc0eb-eb39-465b-b1b2-50bf45bd2854 admin admin] PATH_INFO: `/v3/projects/f119a28bad9944ffa6ae45a86b328ca8` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:04.801770 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71cc0eb-eb39-465b-b1b2-50bf45bd2854 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f119a28bad9944ffa6ae45a86b328ca8)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:04.805286 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71cc0eb-eb39-465b-b1b2-50bf45bd2854 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:04.844041 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1eae35-27c8-4d79-a182-bbaec66fcac8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:04.867015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f59d4a-e528-40a6-b234-b23aa5b456ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:04.869602 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f59d4a-e528-40a6-b234-b23aa5b456ac admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:04.870205 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f59d4a-e528-40a6-b234-b23aa5b456ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:04.870987 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f59d4a-e528-40a6-b234-b23aa5b456ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:04.872454 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f59d4a-e528-40a6-b234-b23aa5b456ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:04.877266 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f59d4a-e528-40a6-b234-b23aa5b456ac admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:04.919322 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 981/1955] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:34:04 2022] POST /identity/v3/projects => generated 367 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:04.921726 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f59d4a-e528-40a6-b234-b23aa5b456ac admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:04.932396 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f71cc0eb-eb39-465b-b1b2-50bf45bd2854 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f119a28bad9944ffa6ae45a86b328ca8', 'request_id': 'req-f71cc0eb-eb39-465b-b1b2-50bf45bd2854'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:04.955010 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f71cc0eb-eb39-465b-b1b2-50bf45bd2854 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f119a28bad9944ffa6ae45a86b328ca8', 'request_id': 'req-f71cc0eb-eb39-465b-b1b2-50bf45bd2854'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:04.973931 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f71cc0eb-eb39-465b-b1b2-50bf45bd2854 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f119a28bad9944ffa6ae45a86b328ca8', 'request_id': 'req-f71cc0eb-eb39-465b-b1b2-50bf45bd2854'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:05.007991 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f71cc0eb-eb39-465b-b1b2-50bf45bd2854 admin admin] The token cache is being invalidate because project f119a28bad9944ffa6ae45a86b328ca8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:05.008560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f71cc0eb-eb39-465b-b1b2-50bf45bd2854 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:05.010681 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f71cc0eb-eb39-465b-b1b2-50bf45bd2854 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:05.013800 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f59d4a-e528-40a6-b234-b23aa5b456ac admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:05.014627 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 975/1956] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:04 2022] DELETE /identity/v3/projects/f119a28bad9944ffa6ae45a86b328ca8 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:05.020513 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feceb732-e015-459f-bdfb-ea6f0f96c967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:05.021285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feceb732-e015-459f-bdfb-ea6f0f96c967 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:05.021643 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feceb732-e015-459f-bdfb-ea6f0f96c967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:05.021780 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4edc9-fd8a-4745-954b-77bc3918b733 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:05.022052 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feceb732-e015-459f-bdfb-ea6f0f96c967 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:05.022363 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4edc9-fd8a-4745-954b-77bc3918b733 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:05.022869 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4edc9-fd8a-4745-954b-77bc3918b733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:05.023010 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feceb732-e015-459f-bdfb-ea6f0f96c967 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:05.025569 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feceb732-e015-459f-bdfb-ea6f0f96c967 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:05.066146 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 982/1957] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:34:04 2022] POST /identity/v3/users => generated 290 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:05.068166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feceb732-e015-459f-bdfb-ea6f0f96c967 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:05.070563 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-feceb732-e015-459f-bdfb-ea6f0f96c967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:05.086172 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feceb732-e015-459f-bdfb-ea6f0f96c967 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:05.089171 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5fb4edc9-fd8a-4745-954b-77bc3918b733 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:05.091798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae1a81-2100-4b2a-9926-e1ed01f7f914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:05.092610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae1a81-2100-4b2a-9926-e1ed01f7f914 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:05.092832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae1a81-2100-4b2a-9926-e1ed01f7f914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:05.093114 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae1a81-2100-4b2a-9926-e1ed01f7f914 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:05.093849 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae1a81-2100-4b2a-9926-e1ed01f7f914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:05.095635 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae1a81-2100-4b2a-9926-e1ed01f7f914 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:05.098740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5fb4edc9-fd8a-4745-954b-77bc3918b733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:05.114469 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 983/1958] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:05 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:05.114469 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae1a81-2100-4b2a-9926-e1ed01f7f914 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:05.137854 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae1a81-2100-4b2a-9926-e1ed01f7f914 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:05.142283 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 976/1959] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:05.144796 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453416b-8c33-4019-bdf3-8c982cd874ad None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:05.145018 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453416b-8c33-4019-bdf3-8c982cd874ad None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:05.145573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453416b-8c33-4019-bdf3-8c982cd874ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:05.159242 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7d406-580f-465c-ad64-3b9d92f2d510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:05.162166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7d406-580f-465c-ad64-3b9d92f2d510 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:05.162741 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7d406-580f-465c-ad64-3b9d92f2d510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:05.164219 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7d406-580f-465c-ad64-3b9d92f2d510 admin admin] PATH_INFO: `/v3/projects/cf14dd9c59ca441ab27e583886c2f44f/users/67d18aaff1e6425dbf49b7b18e4ccb62/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:05.183737 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5453416b-8c33-4019-bdf3-8c982cd874ad None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:05.185018 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5453416b-8c33-4019-bdf3-8c982cd874ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:05.205804 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 977/1960] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:05.211039 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7d406-580f-465c-ad64-3b9d92f2d510 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf14dd9c59ca441ab27e583886c2f44f, user_id=67d18aaff1e6425dbf49b7b18e4ccb62, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:05.211693 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453416b-8c33-4019-bdf3-8c982cd874ad None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:05.214843 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7d406-580f-465c-ad64-3b9d92f2d510 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:05.251164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453416b-8c33-4019-bdf3-8c982cd874ad None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:05.260162 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 984/1961] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:05 2022] PUT /identity/v3/projects/cf14dd9c59ca441ab27e583886c2f44f/users/67d18aaff1e6425dbf49b7b18e4ccb62/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:05.268267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7d406-580f-465c-ad64-3b9d92f2d510 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:05.326387 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:05.367763 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:05.400966 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001e6cb-bfd1-4b18-90d0-2df70c223505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:05.403015 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001e6cb-bfd1-4b18-90d0-2df70c223505 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:05.403408 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001e6cb-bfd1-4b18-90d0-2df70c223505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:05.404445 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b001e6cb-bfd1-4b18-90d0-2df70c223505 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:05.406148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b001e6cb-bfd1-4b18-90d0-2df70c223505 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:34:05.407472 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b001e6cb-bfd1-4b18-90d0-2df70c223505 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:05.412087 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b001e6cb-bfd1-4b18-90d0-2df70c223505 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:05.431986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7d406-580f-465c-ad64-3b9d92f2d510 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:05.437882 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7671a8-9c33-4cef-b919-c6ce16bb4c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:05.438623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7671a8-9c33-4cef-b919-c6ce16bb4c17 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:05.438918 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7671a8-9c33-4cef-b919-c6ce16bb4c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:05.439280 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7671a8-9c33-4cef-b919-c6ce16bb4c17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:05.440333 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7671a8-9c33-4cef-b919-c6ce16bb4c17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:05.442685 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7671a8-9c33-4cef-b919-c6ce16bb4c17 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:05.447215 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 978/1962] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:34:05.451946 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001e6cb-bfd1-4b18-90d0-2df70c223505 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:05.462940 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 985/1963] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:05 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:05.466632 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7671a8-9c33-4cef-b919-c6ce16bb4c17 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:05.489036 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7671a8-9c33-4cef-b919-c6ce16bb4c17 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:05.490259 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b001e6cb-bfd1-4b18-90d0-2df70c223505 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:05.496868 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7780e1-628d-4363-9dd8-8c674e80ddc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:05.497792 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7780e1-628d-4363-9dd8-8c674e80ddc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:05.498072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7780e1-628d-4363-9dd8-8c674e80ddc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:05.498348 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7780e1-628d-4363-9dd8-8c674e80ddc3 admin admin] PATH_INFO: `/v3/projects/cf14dd9c59ca441ab27e583886c2f44f/users/67d18aaff1e6425dbf49b7b18e4ccb62/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:05.499285 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573b01b-8984-4c66-a176-7771434f5e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:05.500334 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573b01b-8984-4c66-a176-7771434f5e46 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:05.500334 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573b01b-8984-4c66-a176-7771434f5e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:05.500533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573b01b-8984-4c66-a176-7771434f5e46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:05.501013 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7780e1-628d-4363-9dd8-8c674e80ddc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf14dd9c59ca441ab27e583886c2f44f, user_id=67d18aaff1e6425dbf49b7b18e4ccb62, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:05.501232 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4573b01b-8984-4c66-a176-7771434f5e46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:05.506378 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4573b01b-8984-4c66-a176-7771434f5e46 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:05.506618 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7780e1-628d-4363-9dd8-8c674e80ddc3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:05.530360 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 986/1964] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:05 2022] PUT /identity/v3/projects/cf14dd9c59ca441ab27e583886c2f44f/users/67d18aaff1e6425dbf49b7b18e4ccb62/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:05.535352 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665868fa-8789-4bf8-a37b-5fde605d9ebd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:05.535865 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665868fa-8789-4bf8-a37b-5fde605d9ebd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:05.536251 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665868fa-8789-4bf8-a37b-5fde605d9ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:05.545022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 979/1965] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:34:05 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:05.550041 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573b01b-8984-4c66-a176-7771434f5e46 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:05.568511 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-665868fa-8789-4bf8-a37b-5fde605d9ebd None None] MFA Rules not processed for user `67d18aaff1e6425dbf49b7b18e4ccb62`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:05.569761 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-665868fa-8789-4bf8-a37b-5fde605d9ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:05.636334 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573b01b-8984-4c66-a176-7771434f5e46 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:05.642615 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77a72c-3e62-4c76-bd5a-b5a12baaf189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:05.642943 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77a72c-3e62-4c76-bd5a-b5a12baaf189 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:05.643365 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77a72c-3e62-4c76-bd5a-b5a12baaf189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:05.644336 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff77a72c-3e62-4c76-bd5a-b5a12baaf189 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:05.645627 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff77a72c-3e62-4c76-bd5a-b5a12baaf189 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:05.647914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff77a72c-3e62-4c76-bd5a-b5a12baaf189 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:05.656709 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 987/1966] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:05 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:34:05.676671 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665868fa-8789-4bf8-a37b-5fde605d9ebd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:05.679115 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-665868fa-8789-4bf8-a37b-5fde605d9ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:05.685325 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 980/1967] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:34:05 2022] POST /identity/v3/users => generated 292 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:34:05.686747 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77a72c-3e62-4c76-bd5a-b5a12baaf189 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:05.699730 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff77a72c-3e62-4c76-bd5a-b5a12baaf189 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:05.707366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95660935-cba9-4bcd-ae37-172038c26362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:05.708386 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95660935-cba9-4bcd-ae37-172038c26362 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:05.708878 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95660935-cba9-4bcd-ae37-172038c26362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:05.709350 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95660935-cba9-4bcd-ae37-172038c26362 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:05.710282 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95660935-cba9-4bcd-ae37-172038c26362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:05.712150 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95660935-cba9-4bcd-ae37-172038c26362 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:05.728118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 981/1968] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:05.732527 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95660935-cba9-4bcd-ae37-172038c26362 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:05.747041 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95660935-cba9-4bcd-ae37-172038c26362 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:05.752450 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b56e82-902d-43f2-8274-53fbfb322907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:05.754106 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b56e82-902d-43f2-8274-53fbfb322907 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:05.754389 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b56e82-902d-43f2-8274-53fbfb322907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:05.754944 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b56e82-902d-43f2-8274-53fbfb322907 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:05.756073 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b56e82-902d-43f2-8274-53fbfb322907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:05.758586 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b56e82-902d-43f2-8274-53fbfb322907 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:05.772963 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665868fa-8789-4bf8-a37b-5fde605d9ebd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:05.774033 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 982/1969] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:05.778330 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b56e82-902d-43f2-8274-53fbfb322907 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:05.779166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d94ef9-7464-4b27-908b-9973d2da2adc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:05.783146 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d94ef9-7464-4b27-908b-9973d2da2adc service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:05.783146 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d94ef9-7464-4b27-908b-9973d2da2adc service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:05.783427 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d94ef9-7464-4b27-908b-9973d2da2adc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:05.792654 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b56e82-902d-43f2-8274-53fbfb322907 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:05.798715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98798c29-7679-4d97-8c8a-e6d08655850c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:05.799996 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98798c29-7679-4d97-8c8a-e6d08655850c admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:05.800362 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98798c29-7679-4d97-8c8a-e6d08655850c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:05.800755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98798c29-7679-4d97-8c8a-e6d08655850c admin admin] PATH_INFO: `/v3/projects/10d55ed8626c4e209d2e584c606a3a3d/users/8df1eeb5c9e143e1a88be1407127c85e/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:05.801903 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d94ef9-7464-4b27-908b-9973d2da2adc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:05.803997 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d94ef9-7464-4b27-908b-9973d2da2adc service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:05.822735 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 988/1970] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:05 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 12:34:05.835795 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98798c29-7679-4d97-8c8a-e6d08655850c admin admin] RBAC: Authorizing `identity:create_grant(project_id=10d55ed8626c4e209d2e584c606a3a3d, user_id=8df1eeb5c9e143e1a88be1407127c85e, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:05.842355 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98798c29-7679-4d97-8c8a-e6d08655850c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:05.870470 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 983/1971] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:05 2022] PUT /identity/v3/projects/10d55ed8626c4e209d2e584c606a3a3d/users/8df1eeb5c9e143e1a88be1407127c85e/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:05.875222 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d94ef9-7464-4b27-908b-9973d2da2adc service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:05.876969 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-55d94ef9-7464-4b27-908b-9973d2da2adc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:05.882231 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428bd4e-d6ba-426d-a57c-41bf8e8ff49e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:05.882659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428bd4e-d6ba-426d-a57c-41bf8e8ff49e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:05.882858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428bd4e-d6ba-426d-a57c-41bf8e8ff49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:05.903934 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d94ef9-7464-4b27-908b-9973d2da2adc service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:05.948982 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:05.965196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a428bd4e-d6ba-426d-a57c-41bf8e8ff49e None None] MFA Rules not processed for user `8df1eeb5c9e143e1a88be1407127c85e`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:05.966517 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a428bd4e-d6ba-426d-a57c-41bf8e8ff49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:06.037394 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434201c-00c7-4ca2-ba82-3ca29c0d7f1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:06.038764 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434201c-00c7-4ca2-ba82-3ca29c0d7f1f service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:06.039052 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434201c-00c7-4ca2-ba82-3ca29c0d7f1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:06.039331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0434201c-00c7-4ca2-ba82-3ca29c0d7f1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:06.042050 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0434201c-00c7-4ca2-ba82-3ca29c0d7f1f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:06.104804 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:06.121951 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 984/1972] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:06.137729 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428bd4e-d6ba-426d-a57c-41bf8e8ff49e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:06.152095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0434201c-00c7-4ca2-ba82-3ca29c0d7f1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:06.153699 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0434201c-00c7-4ca2-ba82-3ca29c0d7f1f service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:06.161312 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428bd4e-d6ba-426d-a57c-41bf8e8ff49e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:06.168587 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 989/1973] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:05 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:06.168725 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70215b4b-de55-43f7-8aaa-0617e6261c39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:06.175049 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70215b4b-de55-43f7-8aaa-0617e6261c39 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:06.177460 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70215b4b-de55-43f7-8aaa-0617e6261c39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:06.177460 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70215b4b-de55-43f7-8aaa-0617e6261c39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:06.186978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70215b4b-de55-43f7-8aaa-0617e6261c39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:06.188582 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70215b4b-de55-43f7-8aaa-0617e6261c39 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:06.210155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 985/1974] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:06.225321 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434201c-00c7-4ca2-ba82-3ca29c0d7f1f service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:06.364044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0434201c-00c7-4ca2-ba82-3ca29c0d7f1f service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:06.372751 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a18a95-0f4b-4bc9-a7bd-ac5e851776d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:06.373844 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a18a95-0f4b-4bc9-a7bd-ac5e851776d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:06.374132 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a18a95-0f4b-4bc9-a7bd-ac5e851776d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:06.374766 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a18a95-0f4b-4bc9-a7bd-ac5e851776d7 admin admin] PATH_INFO: `/v3/users/8df1eeb5c9e143e1a88be1407127c85e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:06.375692 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a18a95-0f4b-4bc9-a7bd-ac5e851776d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8df1eeb5c9e143e1a88be1407127c85e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:06.377784 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a18a95-0f4b-4bc9-a7bd-ac5e851776d7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:06.447509 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e9a18a95-0f4b-4bc9-a7bd-ac5e851776d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8df1eeb5c9e143e1a88be1407127c85e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:06.448446 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e9a18a95-0f4b-4bc9-a7bd-ac5e851776d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8df1eeb5c9e143e1a88be1407127c85e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:06.469193 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e9a18a95-0f4b-4bc9-a7bd-ac5e851776d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8df1eeb5c9e143e1a88be1407127c85e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:06.470110 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e9a18a95-0f4b-4bc9-a7bd-ac5e851776d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8df1eeb5c9e143e1a88be1407127c85e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:06.484623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e9a18a95-0f4b-4bc9-a7bd-ac5e851776d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8df1eeb5c9e143e1a88be1407127c85e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:06.489305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:06.508042 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 990/1975] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:34:06 2022] DELETE /identity/v3/users/8df1eeb5c9e143e1a88be1407127c85e => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:06.510008 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240d6a8-9831-4a8a-af31-97b026282b43 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:06.511403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240d6a8-9831-4a8a-af31-97b026282b43 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:06.511839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240d6a8-9831-4a8a-af31-97b026282b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:06.564431 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6240d6a8-9831-4a8a-af31-97b026282b43 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:06.566578 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6240d6a8-9831-4a8a-af31-97b026282b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:06.715071 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 986/1976] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:06.732684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a18a95-0f4b-4bc9-a7bd-ac5e851776d7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:06.734302 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e9a18a95-0f4b-4bc9-a7bd-ac5e851776d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:06.811972 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a18a95-0f4b-4bc9-a7bd-ac5e851776d7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:06.818104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0137e5-2da5-4315-a375-6e63dbe1f246 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:06.818871 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0137e5-2da5-4315-a375-6e63dbe1f246 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:06.819631 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0137e5-2da5-4315-a375-6e63dbe1f246 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:06.819631 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0137e5-2da5-4315-a375-6e63dbe1f246 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:06.833532 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0137e5-2da5-4315-a375-6e63dbe1f246 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:06.835012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0137e5-2da5-4315-a375-6e63dbe1f246 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:06.846478 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 991/1977] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:07.304734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7467da3-55f9-4202-b8ae-f6b8feb05852 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:07.305257 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7467da3-55f9-4202-b8ae-f6b8feb05852 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:07.305646 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7467da3-55f9-4202-b8ae-f6b8feb05852 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:07.307335 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 987/1978] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jun 29 12:34:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:34:07.324559 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f055bb-bc29-4692-8c05-c06de7346b5f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:07.324559 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f055bb-bc29-4692-8c05-c06de7346b5f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:07.324559 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f055bb-bc29-4692-8c05-c06de7346b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:07.326004 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7467da3-55f9-4202-b8ae-f6b8feb05852 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:07.329401 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d7467da3-55f9-4202-b8ae-f6b8feb05852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:07.366498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d8f055bb-bc29-4692-8c05-c06de7346b5f None None] MFA Rules not processed for user `3ac7391a3c41499d94e5f97478797313`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:07.367893 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d8f055bb-bc29-4692-8c05-c06de7346b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:07.375745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7467da3-55f9-4202-b8ae-f6b8feb05852 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:07.381795 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f952490-46aa-4576-8d2f-15b8534fabe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:07.382702 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f952490-46aa-4576-8d2f-15b8534fabe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:07.382985 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f952490-46aa-4576-8d2f-15b8534fabe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:07.383262 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f952490-46aa-4576-8d2f-15b8534fabe4 admin admin] PATH_INFO: `/v3/projects/10d55ed8626c4e209d2e584c606a3a3d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:07.384501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f952490-46aa-4576-8d2f-15b8534fabe4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10d55ed8626c4e209d2e584c606a3a3d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:07.386448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f952490-46aa-4576-8d2f-15b8534fabe4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:07.464309 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6f952490-46aa-4576-8d2f-15b8534fabe4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10d55ed8626c4e209d2e584c606a3a3d', 'request_id': 'req-6f952490-46aa-4576-8d2f-15b8534fabe4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:07.472275 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6f952490-46aa-4576-8d2f-15b8534fabe4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10d55ed8626c4e209d2e584c606a3a3d', 'request_id': 'req-6f952490-46aa-4576-8d2f-15b8534fabe4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:07.480903 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6f952490-46aa-4576-8d2f-15b8534fabe4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10d55ed8626c4e209d2e584c606a3a3d', 'request_id': 'req-6f952490-46aa-4576-8d2f-15b8534fabe4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:07.498137 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6f952490-46aa-4576-8d2f-15b8534fabe4 admin admin] The token cache is being invalidate because project 10d55ed8626c4e209d2e584c606a3a3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:07.498571 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6f952490-46aa-4576-8d2f-15b8534fabe4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:07.500088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6f952490-46aa-4576-8d2f-15b8534fabe4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:07.500374 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 992/1979] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jun 29 12:34:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:07.501965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 988/1980] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:07 2022] DELETE /identity/v3/projects/10d55ed8626c4e209d2e584c606a3a3d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:07.507586 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a7ec0-2255-4a34-b94c-47d8c46d5c3d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:07.507978 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a7ec0-2255-4a34-b94c-47d8c46d5c3d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:07.508468 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9a7ec0-2255-4a34-b94c-47d8c46d5c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:07.532176 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f952490-46aa-4576-8d2f-15b8534fabe4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:07.532879 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0b9a7ec0-2255-4a34-b94c-47d8c46d5c3d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:07.535237 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0b9a7ec0-2255-4a34-b94c-47d8c46d5c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:07.539914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6f952490-46aa-4576-8d2f-15b8534fabe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:07.660496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 993/1981] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:07.660704 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f952490-46aa-4576-8d2f-15b8534fabe4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:07.662894 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae9ecd-e46e-4236-ab1b-f771850b8802 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:07.663030 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f0ca3-0826-44af-9911-f2995a0359f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:07.663573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f0ca3-0826-44af-9911-f2995a0359f8 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:07.663855 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f0ca3-0826-44af-9911-f2995a0359f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:07.664133 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f0ca3-0826-44af-9911-f2995a0359f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:07.664278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae9ecd-e46e-4236-ab1b-f771850b8802 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:07.664577 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae9ecd-e46e-4236-ab1b-f771850b8802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:07.666170 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d26f0ca3-0826-44af-9911-f2995a0359f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:07.683641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:07.691871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26f0ca3-0826-44af-9911-f2995a0359f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:07.694674 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26f0ca3-0826-44af-9911-f2995a0359f8 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:07.704924 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c1ae9ecd-e46e-4236-ab1b-f771850b8802 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:07.717783 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c1ae9ecd-e46e-4236-ab1b-f771850b8802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:07.728761 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 994/1982] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:07.733656 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 989/1983] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:07.734210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae9ecd-e46e-4236-ab1b-f771850b8802 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:07.747144 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae9ecd-e46e-4236-ab1b-f771850b8802 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:07.753641 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8fac9b-890b-4108-b5d9-aa34ba162ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:07.755195 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8fac9b-890b-4108-b5d9-aa34ba162ae2 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:07.755563 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8fac9b-890b-4108-b5d9-aa34ba162ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:07.755908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8fac9b-890b-4108-b5d9-aa34ba162ae2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:07.756887 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8fac9b-890b-4108-b5d9-aa34ba162ae2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:34:07.757564 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8fac9b-890b-4108-b5d9-aa34ba162ae2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:07.759448 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8fac9b-890b-4108-b5d9-aa34ba162ae2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:07.782020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 995/1984] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:34:07.783677 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f0ca3-0826-44af-9911-f2995a0359f8 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:07.803676 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f0ca3-0826-44af-9911-f2995a0359f8 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:07.818526 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da87f0f-44ce-4d7e-98ce-d4ea93dc943f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:07.822465 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da87f0f-44ce-4d7e-98ce-d4ea93dc943f admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:07.822465 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da87f0f-44ce-4d7e-98ce-d4ea93dc943f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:07.822465 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da87f0f-44ce-4d7e-98ce-d4ea93dc943f admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:07.822465 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da87f0f-44ce-4d7e-98ce-d4ea93dc943f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:07.824139 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da87f0f-44ce-4d7e-98ce-d4ea93dc943f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:07.896108 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 990/1985] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:34:07 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:34:07.901918 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8fac9b-890b-4108-b5d9-aa34ba162ae2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:07.914910 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8fac9b-890b-4108-b5d9-aa34ba162ae2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:07.922795 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:08.010745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e0b06-b413-4b86-8033-f56ffa38db9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:08.013257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e0b06-b413-4b86-8033-f56ffa38db9c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:08.013571 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e0b06-b413-4b86-8033-f56ffa38db9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:08.013857 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e0b06-b413-4b86-8033-f56ffa38db9c admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:08.014612 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e0b06-b413-4b86-8033-f56ffa38db9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:08.016937 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e0b06-b413-4b86-8033-f56ffa38db9c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:08.090204 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 996/1986] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:34:07 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:08.094383 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da87f0f-44ce-4d7e-98ce-d4ea93dc943f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:08.111910 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da87f0f-44ce-4d7e-98ce-d4ea93dc943f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:08.125060 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f167df01-1046-4a27-b41d-e3daf1b6fb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:08.127217 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167df01-1046-4a27-b41d-e3daf1b6fb33 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:08.127510 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167df01-1046-4a27-b41d-e3daf1b6fb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:08.127796 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167df01-1046-4a27-b41d-e3daf1b6fb33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:08.142210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f167df01-1046-4a27-b41d-e3daf1b6fb33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:08.142210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f167df01-1046-4a27-b41d-e3daf1b6fb33 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:08.179914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 991/1987] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:08.181579 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e0b06-b413-4b86-8033-f56ffa38db9c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:08.205469 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e0b06-b413-4b86-8033-f56ffa38db9c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:08.220985 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff1c2f2-0df2-4877-b7cf-853810fd22f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:08.224481 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1c2f2-0df2-4877-b7cf-853810fd22f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:08.224938 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1c2f2-0df2-4877-b7cf-853810fd22f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:08.226247 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff1c2f2-0df2-4877-b7cf-853810fd22f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:08.227711 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff1c2f2-0df2-4877-b7cf-853810fd22f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:08.233285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff1c2f2-0df2-4877-b7cf-853810fd22f6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:08.273566 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 997/1988] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:34:08.274415 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f167df01-1046-4a27-b41d-e3daf1b6fb33 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:08.294940 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f167df01-1046-4a27-b41d-e3daf1b6fb33 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:08.312424 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1a702-c6af-465d-9bfb-9ebca4cedaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:08.316987 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1a702-c6af-465d-9bfb-9ebca4cedaf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:08.317667 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1a702-c6af-465d-9bfb-9ebca4cedaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:08.318843 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1a702-c6af-465d-9bfb-9ebca4cedaf9 admin admin] PATH_INFO: `/v3/projects/b2e6b8e51fc5475ebb09da153aaceaf1/users/aa09e7713f534793ab380c309ed817b7/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:08.364798 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d1a702-c6af-465d-9bfb-9ebca4cedaf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2e6b8e51fc5475ebb09da153aaceaf1, user_id=aa09e7713f534793ab380c309ed817b7, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:08.367838 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d1a702-c6af-465d-9bfb-9ebca4cedaf9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:08.391740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 992/1989] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:08 2022] PUT /identity/v3/projects/b2e6b8e51fc5475ebb09da153aaceaf1/users/aa09e7713f534793ab380c309ed817b7/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:34:08.402772 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae67edf-c8d6-4638-ba1c-9fd1c38a252a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:08.403323 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae67edf-c8d6-4638-ba1c-9fd1c38a252a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:08.403402 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae67edf-c8d6-4638-ba1c-9fd1c38a252a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:08.471193 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2ae67edf-c8d6-4638-ba1c-9fd1c38a252a None None] MFA Rules not processed for user `aa09e7713f534793ab380c309ed817b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:08.472922 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2ae67edf-c8d6-4638-ba1c-9fd1c38a252a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:08.593044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 998/1990] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:08.606667 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1a702-c6af-465d-9bfb-9ebca4cedaf9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:08.608870 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e6d1a702-c6af-465d-9bfb-9ebca4cedaf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:08.698028 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1a702-c6af-465d-9bfb-9ebca4cedaf9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:08.702269 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5e064-141f-42b0-bc6a-132b77cac632 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:08.703040 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5e064-141f-42b0-bc6a-132b77cac632 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:08.703326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5e064-141f-42b0-bc6a-132b77cac632 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:08.703606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5e064-141f-42b0-bc6a-132b77cac632 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:08.720159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5e064-141f-42b0-bc6a-132b77cac632 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:08.722735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d5e064-141f-42b0-bc6a-132b77cac632 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:08.733134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 993/1991] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:10.287799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae67edf-c8d6-4638-ba1c-9fd1c38a252a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:10.289297 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2ae67edf-c8d6-4638-ba1c-9fd1c38a252a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:10.470043 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae67edf-c8d6-4638-ba1c-9fd1c38a252a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:10.477649 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c3ba16-7899-4412-a03c-8858057157d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:10.479088 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3ba16-7899-4412-a03c-8858057157d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:10.479634 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3ba16-7899-4412-a03c-8858057157d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:10.480284 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3ba16-7899-4412-a03c-8858057157d8 admin admin] PATH_INFO: `/v3/users/6f1324e1021a473f8f0044b2ca28234c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:10.482045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c3ba16-7899-4412-a03c-8858057157d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f1324e1021a473f8f0044b2ca28234c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:10.486321 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c3ba16-7899-4412-a03c-8858057157d8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:10.521050 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:10.574888 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c4c3ba16-7899-4412-a03c-8858057157d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f1324e1021a473f8f0044b2ca28234c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:10.575914 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c4c3ba16-7899-4412-a03c-8858057157d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f1324e1021a473f8f0044b2ca28234c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:10.597559 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c4c3ba16-7899-4412-a03c-8858057157d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f1324e1021a473f8f0044b2ca28234c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:10.598308 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c4c3ba16-7899-4412-a03c-8858057157d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f1324e1021a473f8f0044b2ca28234c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:10.615454 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c4c3ba16-7899-4412-a03c-8858057157d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f1324e1021a473f8f0044b2ca28234c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:10.634485 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 999/1992] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:34:10 2022] DELETE /identity/v3/users/6f1324e1021a473f8f0044b2ca28234c => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:10.637904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aab7fa-c473-45bc-829e-ed5ad68446af None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:10.638476 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aab7fa-c473-45bc-829e-ed5ad68446af None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:10.638588 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aab7fa-c473-45bc-829e-ed5ad68446af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:10.658263 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d1aab7fa-c473-45bc-829e-ed5ad68446af None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:10.660051 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d1aab7fa-c473-45bc-829e-ed5ad68446af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:10.734179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 994/1993] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:10.745984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c3ba16-7899-4412-a03c-8858057157d8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:10.747406 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c4c3ba16-7899-4412-a03c-8858057157d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:10.806925 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c3ba16-7899-4412-a03c-8858057157d8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:10.810537 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e1477-5dc0-4228-846f-69051964288a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:10.811186 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e1477-5dc0-4228-846f-69051964288a service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:10.811471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e1477-5dc0-4228-846f-69051964288a service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:10.811749 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e1477-5dc0-4228-846f-69051964288a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:10.820931 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730e1477-5dc0-4228-846f-69051964288a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:10.822356 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730e1477-5dc0-4228-846f-69051964288a service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:10.832924 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1000/1994] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:11.220396 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aab7fa-c473-45bc-829e-ed5ad68446af None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:11.222278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d1aab7fa-c473-45bc-829e-ed5ad68446af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:11.234554 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aab7fa-c473-45bc-829e-ed5ad68446af None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:11.239468 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc2ef7-b08a-4523-8619-af81f6aaec22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:11.240334 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc2ef7-b08a-4523-8619-af81f6aaec22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:11.240623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc2ef7-b08a-4523-8619-af81f6aaec22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:11.240903 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc2ef7-b08a-4523-8619-af81f6aaec22 admin admin] PATH_INFO: `/v3/projects/329fe57379ac4f89a92e0394d3a99342` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:11.242389 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dc2ef7-b08a-4523-8619-af81f6aaec22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=329fe57379ac4f89a92e0394d3a99342)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:11.244212 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dc2ef7-b08a-4523-8619-af81f6aaec22 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:11.295421 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:11.310502 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-72dc2ef7-b08a-4523-8619-af81f6aaec22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '329fe57379ac4f89a92e0394d3a99342', 'request_id': 'req-72dc2ef7-b08a-4523-8619-af81f6aaec22'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:11.319045 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-72dc2ef7-b08a-4523-8619-af81f6aaec22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '329fe57379ac4f89a92e0394d3a99342', 'request_id': 'req-72dc2ef7-b08a-4523-8619-af81f6aaec22'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:11.327091 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-72dc2ef7-b08a-4523-8619-af81f6aaec22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '329fe57379ac4f89a92e0394d3a99342', 'request_id': 'req-72dc2ef7-b08a-4523-8619-af81f6aaec22'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:11.346499 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-72dc2ef7-b08a-4523-8619-af81f6aaec22 admin admin] The token cache is being invalidate because project 329fe57379ac4f89a92e0394d3a99342 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:11.346794 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-72dc2ef7-b08a-4523-8619-af81f6aaec22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:11.347445 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-72dc2ef7-b08a-4523-8619-af81f6aaec22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:11.349873 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 995/1995] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:11 2022] DELETE /identity/v3/projects/329fe57379ac4f89a92e0394d3a99342 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:11.353910 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e1477-5dc0-4228-846f-69051964288a service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:11.365693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e1477-5dc0-4228-846f-69051964288a service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:11.440181 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0a4c9-1725-42d9-b5d7-d8d40b3b6fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:11.441121 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0a4c9-1725-42d9-b5d7-d8d40b3b6fc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:11.441403 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0a4c9-1725-42d9-b5d7-d8d40b3b6fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:11.441682 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0a4c9-1725-42d9-b5d7-d8d40b3b6fc9 admin admin] PATH_INFO: `/v3/users/67d18aaff1e6425dbf49b7b18e4ccb62` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:11.442946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df0a4c9-1725-42d9-b5d7-d8d40b3b6fc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67d18aaff1e6425dbf49b7b18e4ccb62)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:11.444870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df0a4c9-1725-42d9-b5d7-d8d40b3b6fc9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:11.487709 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:11.494434 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9df0a4c9-1725-42d9-b5d7-d8d40b3b6fc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67d18aaff1e6425dbf49b7b18e4ccb62'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:11.496346 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9df0a4c9-1725-42d9-b5d7-d8d40b3b6fc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67d18aaff1e6425dbf49b7b18e4ccb62'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:11.511010 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9df0a4c9-1725-42d9-b5d7-d8d40b3b6fc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67d18aaff1e6425dbf49b7b18e4ccb62'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:11.511734 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9df0a4c9-1725-42d9-b5d7-d8d40b3b6fc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67d18aaff1e6425dbf49b7b18e4ccb62'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:11.519098 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9df0a4c9-1725-42d9-b5d7-d8d40b3b6fc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67d18aaff1e6425dbf49b7b18e4ccb62'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:11.532643 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1001/1996] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:34:11 2022] DELETE /identity/v3/users/67d18aaff1e6425dbf49b7b18e4ccb62 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:11.966502 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc2ef7-b08a-4523-8619-af81f6aaec22 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:11.968680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-72dc2ef7-b08a-4523-8619-af81f6aaec22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:11.984696 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc2ef7-b08a-4523-8619-af81f6aaec22 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:12.052868 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6075380f-fe47-4ffd-b5ed-d3550dfdd5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:12.053745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075380f-fe47-4ffd-b5ed-d3550dfdd5ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:12.054127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075380f-fe47-4ffd-b5ed-d3550dfdd5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:12.054409 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6075380f-fe47-4ffd-b5ed-d3550dfdd5ef admin admin] PATH_INFO: `/v3/projects/cf14dd9c59ca441ab27e583886c2f44f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:12.055629 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6075380f-fe47-4ffd-b5ed-d3550dfdd5ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf14dd9c59ca441ab27e583886c2f44f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:12.058102 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6075380f-fe47-4ffd-b5ed-d3550dfdd5ef admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:12.158221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6075380f-fe47-4ffd-b5ed-d3550dfdd5ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf14dd9c59ca441ab27e583886c2f44f', 'request_id': 'req-6075380f-fe47-4ffd-b5ed-d3550dfdd5ef'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:12.167734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6075380f-fe47-4ffd-b5ed-d3550dfdd5ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf14dd9c59ca441ab27e583886c2f44f', 'request_id': 'req-6075380f-fe47-4ffd-b5ed-d3550dfdd5ef'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:12.176397 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6075380f-fe47-4ffd-b5ed-d3550dfdd5ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf14dd9c59ca441ab27e583886c2f44f', 'request_id': 'req-6075380f-fe47-4ffd-b5ed-d3550dfdd5ef'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:12.193659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6075380f-fe47-4ffd-b5ed-d3550dfdd5ef admin admin] The token cache is being invalidate because project cf14dd9c59ca441ab27e583886c2f44f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:12.194040 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6075380f-fe47-4ffd-b5ed-d3550dfdd5ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:12.194908 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6075380f-fe47-4ffd-b5ed-d3550dfdd5ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:12.199989 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 996/1997] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:11 2022] DELETE /identity/v3/projects/cf14dd9c59ca441ab27e583886c2f44f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:12.201811 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036441a-d8c2-468b-8cd7-53f12c73d53a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:12.202149 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036441a-d8c2-468b-8cd7-53f12c73d53a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:12.202496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9036441a-d8c2-468b-8cd7-53f12c73d53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:12.227361 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9036441a-d8c2-468b-8cd7-53f12c73d53a None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:12.228414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9036441a-d8c2-468b-8cd7-53f12c73d53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:12.321388 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1002/1998] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:12.329435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce19bf-201f-441a-864e-ce5e2a30451c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:12.329726 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce19bf-201f-441a-864e-ce5e2a30451c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:12.330005 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ce19bf-201f-441a-864e-ce5e2a30451c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:12.350591 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b7ce19bf-201f-441a-864e-ce5e2a30451c None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:12.351932 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b7ce19bf-201f-441a-864e-ce5e2a30451c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:12.360285 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 997/1999] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:12.364610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036441a-d8c2-468b-8cd7-53f12c73d53a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:12.374540 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9036441a-d8c2-468b-8cd7-53f12c73d53a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:12.379358 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d55e521-5ff1-4123-bd01-24ee74038823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:12.380017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55e521-5ff1-4123-bd01-24ee74038823 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:12.380302 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55e521-5ff1-4123-bd01-24ee74038823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:12.380577 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55e521-5ff1-4123-bd01-24ee74038823 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:12.381302 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d55e521-5ff1-4123-bd01-24ee74038823 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:34:12.381607 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d55e521-5ff1-4123-bd01-24ee74038823 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:12.383268 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d55e521-5ff1-4123-bd01-24ee74038823 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:12.397042 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1003/2000] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:34:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:34:12.401344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce19bf-201f-441a-864e-ce5e2a30451c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:12.411911 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ce19bf-201f-441a-864e-ce5e2a30451c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:12.416587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529a3c5-3c01-48bc-8d37-9b82f7941436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:12.417238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529a3c5-3c01-48bc-8d37-9b82f7941436 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:12.417524 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529a3c5-3c01-48bc-8d37-9b82f7941436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:12.417801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529a3c5-3c01-48bc-8d37-9b82f7941436 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:12.418514 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8529a3c5-3c01-48bc-8d37-9b82f7941436 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:12.420458 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8529a3c5-3c01-48bc-8d37-9b82f7941436 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:12.453682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 998/2001] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:34:12 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:12.457835 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d55e521-5ff1-4123-bd01-24ee74038823 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:12.468885 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d55e521-5ff1-4123-bd01-24ee74038823 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:12.533345 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4cf5f5-a21f-405f-94f3-4b7e88047f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:12.533706 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4cf5f5-a21f-405f-94f3-4b7e88047f50 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:12.533998 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4cf5f5-a21f-405f-94f3-4b7e88047f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:12.536085 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4cf5f5-a21f-405f-94f3-4b7e88047f50 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:12.536966 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4cf5f5-a21f-405f-94f3-4b7e88047f50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:12.539417 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4cf5f5-a21f-405f-94f3-4b7e88047f50 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:12.561615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1004/2002] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:34:12 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:12.565627 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529a3c5-3c01-48bc-8d37-9b82f7941436 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:12.580788 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529a3c5-3c01-48bc-8d37-9b82f7941436 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:12.585596 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa24c8-49dc-4731-ab90-9b5a4848077f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:12.586273 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa24c8-49dc-4731-ab90-9b5a4848077f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:12.586593 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa24c8-49dc-4731-ab90-9b5a4848077f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:12.587270 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa24c8-49dc-4731-ab90-9b5a4848077f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:12.587594 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa24c8-49dc-4731-ab90-9b5a4848077f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:12.589202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aa24c8-49dc-4731-ab90-9b5a4848077f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:12.603923 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 999/2003] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:12.605145 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4cf5f5-a21f-405f-94f3-4b7e88047f50 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:12.617644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4cf5f5-a21f-405f-94f3-4b7e88047f50 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:12.622353 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09f004-62c3-4f3c-986e-223073e29ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:12.623084 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09f004-62c3-4f3c-986e-223073e29ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:12.623373 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09f004-62c3-4f3c-986e-223073e29ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:12.623656 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09f004-62c3-4f3c-986e-223073e29ff8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:12.624391 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09f004-62c3-4f3c-986e-223073e29ff8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:12.626024 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09f004-62c3-4f3c-986e-223073e29ff8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:12.637038 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1005/2004] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:12.641143 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa24c8-49dc-4731-ab90-9b5a4848077f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:12.652881 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa24c8-49dc-4731-ab90-9b5a4848077f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:12.662249 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea285d7-84de-430f-b52e-1d84c7a1deb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:12.663181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea285d7-84de-430f-b52e-1d84c7a1deb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:12.663460 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea285d7-84de-430f-b52e-1d84c7a1deb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:12.663737 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea285d7-84de-430f-b52e-1d84c7a1deb3 admin admin] PATH_INFO: `/v3/projects/a0b563675dfa48ab8f91309a4b2d96c4/users/020e59bdec714f3e90a0a3d1fe9e9efe/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:12.676564 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:12.679816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea285d7-84de-430f-b52e-1d84c7a1deb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0b563675dfa48ab8f91309a4b2d96c4, user_id=020e59bdec714f3e90a0a3d1fe9e9efe, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:12.682895 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea285d7-84de-430f-b52e-1d84c7a1deb3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:12.700203 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1000/2005] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:12 2022] PUT /identity/v3/projects/a0b563675dfa48ab8f91309a4b2d96c4/users/020e59bdec714f3e90a0a3d1fe9e9efe/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:12.711349 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0256a6c-c43f-446a-b863-55fbc108d9ce None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:12.711642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0256a6c-c43f-446a-b863-55fbc108d9ce None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:12.711925 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0256a6c-c43f-446a-b863-55fbc108d9ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:12.746727 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c0256a6c-c43f-446a-b863-55fbc108d9ce None None] MFA Rules not processed for user `020e59bdec714f3e90a0a3d1fe9e9efe`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:12.748091 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c0256a6c-c43f-446a-b863-55fbc108d9ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:12.761086 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:12.830684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1006/2006] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:34:12.835534 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea285d7-84de-430f-b52e-1d84c7a1deb3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:12.928019 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea285d7-84de-430f-b52e-1d84c7a1deb3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:12.933623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870fc7f4-14a3-4a97-85e0-9c3d7d809a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:12.934121 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870fc7f4-14a3-4a97-85e0-9c3d7d809a1b admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:12.934508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870fc7f4-14a3-4a97-85e0-9c3d7d809a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:12.934862 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870fc7f4-14a3-4a97-85e0-9c3d7d809a1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:12.935655 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870fc7f4-14a3-4a97-85e0-9c3d7d809a1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:12.937723 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870fc7f4-14a3-4a97-85e0-9c3d7d809a1b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:12.975853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1001/2007] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:34:12 2022] POST /identity/v3/projects => generated 381 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:12.980134 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0256a6c-c43f-446a-b863-55fbc108d9ce None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:12.995571 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0256a6c-c43f-446a-b863-55fbc108d9ce None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:13.062461 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14f241-1a91-438b-aa6e-df9f0a822665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:13.062814 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14f241-1a91-438b-aa6e-df9f0a822665 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:13.062952 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14f241-1a91-438b-aa6e-df9f0a822665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:13.063237 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14f241-1a91-438b-aa6e-df9f0a822665 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:13.063944 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14f241-1a91-438b-aa6e-df9f0a822665 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:13.066543 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14f241-1a91-438b-aa6e-df9f0a822665 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:13.090853 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1007/2008] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:34:12 2022] POST /identity/v3/users => generated 297 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:13.094960 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870fc7f4-14a3-4a97-85e0-9c3d7d809a1b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:13.108171 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870fc7f4-14a3-4a97-85e0-9c3d7d809a1b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:13.114586 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba1f9b-60b9-4784-aff6-28a4bbafd697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:13.115268 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba1f9b-60b9-4784-aff6-28a4bbafd697 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:13.115550 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba1f9b-60b9-4784-aff6-28a4bbafd697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:13.115825 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba1f9b-60b9-4784-aff6-28a4bbafd697 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:13.116557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba1f9b-60b9-4784-aff6-28a4bbafd697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:13.118158 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba1f9b-60b9-4784-aff6-28a4bbafd697 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:13.130022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1002/2009] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:13.135501 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14f241-1a91-438b-aa6e-df9f0a822665 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:13.151649 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14f241-1a91-438b-aa6e-df9f0a822665 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:13.152176 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9ff5b-7773-4fcd-a873-08f5b1d027ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:13.153090 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9ff5b-7773-4fcd-a873-08f5b1d027ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:13.153387 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9ff5b-7773-4fcd-a873-08f5b1d027ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:13.153678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c9ff5b-7773-4fcd-a873-08f5b1d027ee admin admin] PATH_INFO: `/v3/projects/a1ba1b7d2d2a476bb12ea1266ac98afc/users/105c734364114845a48241279a2c7d68/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:13.171498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9ff5b-7773-4fcd-a873-08f5b1d027ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1ba1b7d2d2a476bb12ea1266ac98afc, user_id=105c734364114845a48241279a2c7d68, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:13.174200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c9ff5b-7773-4fcd-a873-08f5b1d027ee admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:13.193324 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1008/2010] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:13 2022] PUT /identity/v3/projects/a1ba1b7d2d2a476bb12ea1266ac98afc/users/105c734364114845a48241279a2c7d68/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:13.198445 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba1f9b-60b9-4784-aff6-28a4bbafd697 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:13.215574 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba1f9b-60b9-4784-aff6-28a4bbafd697 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:13.281691 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aace542b-bd4f-4608-87e3-18597f2f6efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:13.282390 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace542b-bd4f-4608-87e3-18597f2f6efb admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:13.282699 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace542b-bd4f-4608-87e3-18597f2f6efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:13.282981 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace542b-bd4f-4608-87e3-18597f2f6efb admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:13.283697 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aace542b-bd4f-4608-87e3-18597f2f6efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:13.285933 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aace542b-bd4f-4608-87e3-18597f2f6efb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:13.298266 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1003/2011] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:13 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:13.300326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9ff5b-7773-4fcd-a873-08f5b1d027ee admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:13.311122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c9ff5b-7773-4fcd-a873-08f5b1d027ee admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:13.318076 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b7674-6709-4d07-b9fb-136aedcded70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:13.319082 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b7674-6709-4d07-b9fb-136aedcded70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:13.319440 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b7674-6709-4d07-b9fb-136aedcded70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:13.319788 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b7674-6709-4d07-b9fb-136aedcded70 admin admin] PATH_INFO: `/v3/projects/a1ba1b7d2d2a476bb12ea1266ac98afc/users/105c734364114845a48241279a2c7d68/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:13.322266 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6b7674-6709-4d07-b9fb-136aedcded70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1ba1b7d2d2a476bb12ea1266ac98afc, user_id=105c734364114845a48241279a2c7d68, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:13.325087 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6b7674-6709-4d07-b9fb-136aedcded70 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:13.338414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1009/2012] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:13 2022] PUT /identity/v3/projects/a1ba1b7d2d2a476bb12ea1266ac98afc/users/105c734364114845a48241279a2c7d68/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:13.350838 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df667b-33d0-4329-bdb0-c355fe5d6c65 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:13.351135 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df667b-33d0-4329-bdb0-c355fe5d6c65 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:13.351425 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df667b-33d0-4329-bdb0-c355fe5d6c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:13.395086 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-37df667b-33d0-4329-bdb0-c355fe5d6c65 None None] MFA Rules not processed for user `105c734364114845a48241279a2c7d68`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:13.396659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-37df667b-33d0-4329-bdb0-c355fe5d6c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:13.458164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:13.493097 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1004/2013] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:13.505951 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b7674-6709-4d07-b9fb-136aedcded70 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:13.508164 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9b6b7674-6709-4d07-b9fb-136aedcded70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:13.569433 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b7674-6709-4d07-b9fb-136aedcded70 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:13.573048 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef598e-87b6-46c4-bd12-8c06be4ac867 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:13.573688 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef598e-87b6-46c4-bd12-8c06be4ac867 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:13.573969 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef598e-87b6-46c4-bd12-8c06be4ac867 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:13.574244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef598e-87b6-46c4-bd12-8c06be4ac867 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:13.583614 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feef598e-87b6-46c4-bd12-8c06be4ac867 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:13.585031 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feef598e-87b6-46c4-bd12-8c06be4ac867 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:13.594253 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1010/2014] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:13 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:13.936093 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df667b-33d0-4329-bdb0-c355fe5d6c65 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:13.947824 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df667b-33d0-4329-bdb0-c355fe5d6c65 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:13.952608 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f464e6-33b9-4c0c-98cc-6bfeec7e2098 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105c734364114845a48241279a2c7d68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1ba1b7d2d2a476bb12ea1266ac98afc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:13.953333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f464e6-33b9-4c0c-98cc-6bfeec7e2098 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:13.953627 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f464e6-33b9-4c0c-98cc-6bfeec7e2098 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:13.954241 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f464e6-33b9-4c0c-98cc-6bfeec7e2098 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:13.955063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f464e6-33b9-4c0c-98cc-6bfeec7e2098 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:34:13.955384 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f464e6-33b9-4c0c-98cc-6bfeec7e2098 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:13.957067 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f464e6-33b9-4c0c-98cc-6bfeec7e2098 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:13.973517 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1005/2015] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:34:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:34:13.974694 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef598e-87b6-46c4-bd12-8c06be4ac867 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:13.985789 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef598e-87b6-46c4-bd12-8c06be4ac867 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:13.991657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6641e6-eeae-4c51-a10d-6dbcff9b6c27 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105c734364114845a48241279a2c7d68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1ba1b7d2d2a476bb12ea1266ac98afc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:13.992343 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6641e6-eeae-4c51-a10d-6dbcff9b6c27 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:13.992669 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6641e6-eeae-4c51-a10d-6dbcff9b6c27 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:13.992956 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6641e6-eeae-4c51-a10d-6dbcff9b6c27 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:13.993674 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6641e6-eeae-4c51-a10d-6dbcff9b6c27 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:13.995650 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6641e6-eeae-4c51-a10d-6dbcff9b6c27 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:14.000194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:14.026789 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1011/2016] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:34:13 2022] POST /identity/v3/projects => generated 353 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:34:14.030760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f464e6-33b9-4c0c-98cc-6bfeec7e2098 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:14.043356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f464e6-33b9-4c0c-98cc-6bfeec7e2098 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:14.133500 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3441e-5ae6-4cab-8871-2108ae55140a tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105c734364114845a48241279a2c7d68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1ba1b7d2d2a476bb12ea1266ac98afc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:14.134141 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3441e-5ae6-4cab-8871-2108ae55140a tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:14.134627 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3441e-5ae6-4cab-8871-2108ae55140a tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:14.134912 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b3441e-5ae6-4cab-8871-2108ae55140a tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:14.135633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b3441e-5ae6-4cab-8871-2108ae55140a tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:34:14.135945 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b3441e-5ae6-4cab-8871-2108ae55140a tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:14.137550 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b3441e-5ae6-4cab-8871-2108ae55140a tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:14.162940 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1006/2017] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:34:14.210537 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6641e6-eeae-4c51-a10d-6dbcff9b6c27 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:14.233753 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6641e6-eeae-4c51-a10d-6dbcff9b6c27 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:14.248541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd669d5-e88e-476a-bfc9-4229ed38f5f0 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '105c734364114845a48241279a2c7d68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a1ba1b7d2d2a476bb12ea1266ac98afc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:14.250091 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd669d5-e88e-476a-bfc9-4229ed38f5f0 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:14.250814 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd669d5-e88e-476a-bfc9-4229ed38f5f0 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:14.251306 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd669d5-e88e-476a-bfc9-4229ed38f5f0 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] PATH_INFO: `/v3/projects/4ed92aad3f424be0bcfafd3a4122b33b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:14.254283 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd669d5-e88e-476a-bfc9-4229ed38f5f0 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4ed92aad3f424be0bcfafd3a4122b33b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:14.256434 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd669d5-e88e-476a-bfc9-4229ed38f5f0 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:14.312039 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7fd669d5-e88e-476a-bfc9-4229ed38f5f0 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ed92aad3f424be0bcfafd3a4122b33b', 'request_id': 'req-7fd669d5-e88e-476a-bfc9-4229ed38f5f0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:14.320283 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7fd669d5-e88e-476a-bfc9-4229ed38f5f0 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ed92aad3f424be0bcfafd3a4122b33b', 'request_id': 'req-7fd669d5-e88e-476a-bfc9-4229ed38f5f0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:14.328239 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7fd669d5-e88e-476a-bfc9-4229ed38f5f0 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ed92aad3f424be0bcfafd3a4122b33b', 'request_id': 'req-7fd669d5-e88e-476a-bfc9-4229ed38f5f0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:14.340533 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7fd669d5-e88e-476a-bfc9-4229ed38f5f0 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] The token cache is being invalidate because project 4ed92aad3f424be0bcfafd3a4122b33b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:14.340842 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7fd669d5-e88e-476a-bfc9-4229ed38f5f0 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:14.341505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7fd669d5-e88e-476a-bfc9-4229ed38f5f0 tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:14.346414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1012/2018] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:14 2022] DELETE /identity/v3/projects/4ed92aad3f424be0bcfafd3a4122b33b => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:14.550465 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3441e-5ae6-4cab-8871-2108ae55140a tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:14.552418 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-97b3441e-5ae6-4cab-8871-2108ae55140a tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:14.633226 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b3441e-5ae6-4cab-8871-2108ae55140a tempest-VolumeQuotaClassesTest-1922630531 tempest-VolumeQuotaClassesTest-1922630531-project-admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:14.697566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bff949-5789-40bf-82be-a58433da9636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:14.698321 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bff949-5789-40bf-82be-a58433da9636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:14.698638 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bff949-5789-40bf-82be-a58433da9636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:14.699373 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bff949-5789-40bf-82be-a58433da9636 admin admin] PATH_INFO: `/v3/users/020e59bdec714f3e90a0a3d1fe9e9efe` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:14.700188 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bff949-5789-40bf-82be-a58433da9636 admin admin] RBAC: Authorizing `identity:delete_user(user_id=020e59bdec714f3e90a0a3d1fe9e9efe)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:14.702244 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bff949-5789-40bf-82be-a58433da9636 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:14.718149 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:14.771698 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d3bff949-5789-40bf-82be-a58433da9636 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '020e59bdec714f3e90a0a3d1fe9e9efe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:14.782138 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d3bff949-5789-40bf-82be-a58433da9636 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '020e59bdec714f3e90a0a3d1fe9e9efe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:14.801160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d3bff949-5789-40bf-82be-a58433da9636 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '020e59bdec714f3e90a0a3d1fe9e9efe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:14.801726 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d3bff949-5789-40bf-82be-a58433da9636 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '020e59bdec714f3e90a0a3d1fe9e9efe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:14.802339 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d3bff949-5789-40bf-82be-a58433da9636 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '020e59bdec714f3e90a0a3d1fe9e9efe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:14.820890 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1007/2019] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:34:14 2022] DELETE /identity/v3/users/020e59bdec714f3e90a0a3d1fe9e9efe => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:14.825818 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d0beb-f8bb-4bea-bc05-8a0eed2432fe None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:14.826332 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d0beb-f8bb-4bea-bc05-8a0eed2432fe None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:14.826493 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d0beb-f8bb-4bea-bc05-8a0eed2432fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:14.846866 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e28d0beb-f8bb-4bea-bc05-8a0eed2432fe None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:14.848416 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e28d0beb-f8bb-4bea-bc05-8a0eed2432fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:14.956826 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1013/2020] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:34:14.967369 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bff949-5789-40bf-82be-a58433da9636 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:14.968931 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d3bff949-5789-40bf-82be-a58433da9636 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:15.036181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bff949-5789-40bf-82be-a58433da9636 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:15.040941 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cec7ff-7cd5-402a-8c1e-de7ebb5ef77c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:15.041968 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cec7ff-7cd5-402a-8c1e-de7ebb5ef77c service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:15.042407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cec7ff-7cd5-402a-8c1e-de7ebb5ef77c service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:15.042715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cec7ff-7cd5-402a-8c1e-de7ebb5ef77c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:15.053046 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cec7ff-7cd5-402a-8c1e-de7ebb5ef77c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:15.054543 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cec7ff-7cd5-402a-8c1e-de7ebb5ef77c service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:15.069505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1008/2021] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:15.376037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d0beb-f8bb-4bea-bc05-8a0eed2432fe None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:15.377481 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e28d0beb-f8bb-4bea-bc05-8a0eed2432fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:15.396464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d0beb-f8bb-4bea-bc05-8a0eed2432fe None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:15.407418 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9644fe-97b0-44be-be88-71868a7ed6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:15.408598 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9644fe-97b0-44be-be88-71868a7ed6d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:15.409109 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9644fe-97b0-44be-be88-71868a7ed6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:15.409492 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9644fe-97b0-44be-be88-71868a7ed6d7 admin admin] PATH_INFO: `/v3/projects/a0b563675dfa48ab8f91309a4b2d96c4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:15.411349 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9644fe-97b0-44be-be88-71868a7ed6d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0b563675dfa48ab8f91309a4b2d96c4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:15.414506 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9644fe-97b0-44be-be88-71868a7ed6d7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:15.479646 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-da9644fe-97b0-44be-be88-71868a7ed6d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0b563675dfa48ab8f91309a4b2d96c4', 'request_id': 'req-da9644fe-97b0-44be-be88-71868a7ed6d7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:15.487017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-da9644fe-97b0-44be-be88-71868a7ed6d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0b563675dfa48ab8f91309a4b2d96c4', 'request_id': 'req-da9644fe-97b0-44be-be88-71868a7ed6d7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:15.494130 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-da9644fe-97b0-44be-be88-71868a7ed6d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0b563675dfa48ab8f91309a4b2d96c4', 'request_id': 'req-da9644fe-97b0-44be-be88-71868a7ed6d7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:15.498186 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=84,keystone:INSERT=3,keystone:DELETE=13 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:15.507276 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-da9644fe-97b0-44be-be88-71868a7ed6d7 admin admin] The token cache is being invalidate because project a0b563675dfa48ab8f91309a4b2d96c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:15.507925 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-da9644fe-97b0-44be-be88-71868a7ed6d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:15.509341 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-da9644fe-97b0-44be-be88-71868a7ed6d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:15.511420 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1014/2022] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:15 2022] DELETE /identity/v3/projects/a0b563675dfa48ab8f91309a4b2d96c4 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:15.515330 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cec7ff-7cd5-402a-8c1e-de7ebb5ef77c service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:15.525226 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cec7ff-7cd5-402a-8c1e-de7ebb5ef77c service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:15.582744 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1210dd3-771c-4fb4-aa10-b9f8dc66f0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:15.583610 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1210dd3-771c-4fb4-aa10-b9f8dc66f0ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:15.583897 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1210dd3-771c-4fb4-aa10-b9f8dc66f0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:15.584178 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1210dd3-771c-4fb4-aa10-b9f8dc66f0ff admin admin] PATH_INFO: `/v3/users/105c734364114845a48241279a2c7d68` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:15.585354 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1210dd3-771c-4fb4-aa10-b9f8dc66f0ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=105c734364114845a48241279a2c7d68)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:15.587335 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1210dd3-771c-4fb4-aa10-b9f8dc66f0ff admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:15.627123 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:15.638402 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b1210dd3-771c-4fb4-aa10-b9f8dc66f0ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '105c734364114845a48241279a2c7d68'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:15.646980 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b1210dd3-771c-4fb4-aa10-b9f8dc66f0ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '105c734364114845a48241279a2c7d68'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:15.660306 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b1210dd3-771c-4fb4-aa10-b9f8dc66f0ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '105c734364114845a48241279a2c7d68'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:15.660957 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b1210dd3-771c-4fb4-aa10-b9f8dc66f0ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '105c734364114845a48241279a2c7d68'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:15.661590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b1210dd3-771c-4fb4-aa10-b9f8dc66f0ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '105c734364114845a48241279a2c7d68'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:15.673730 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1009/2023] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:34:15 2022] DELETE /identity/v3/users/105c734364114845a48241279a2c7d68 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:15.955204 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9644fe-97b0-44be-be88-71868a7ed6d7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:15.956932 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-da9644fe-97b0-44be-be88-71868a7ed6d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:15.972011 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9644fe-97b0-44be-be88-71868a7ed6d7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:16.036612 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07f4dd-7c1f-427d-8609-975c02559864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:16.037485 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07f4dd-7c1f-427d-8609-975c02559864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:16.037768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07f4dd-7c1f-427d-8609-975c02559864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:16.038040 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07f4dd-7c1f-427d-8609-975c02559864 admin admin] PATH_INFO: `/v3/projects/a1ba1b7d2d2a476bb12ea1266ac98afc` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:16.039303 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e07f4dd-7c1f-427d-8609-975c02559864 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1ba1b7d2d2a476bb12ea1266ac98afc)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:16.041356 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e07f4dd-7c1f-427d-8609-975c02559864 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:16.090151 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3e07f4dd-7c1f-427d-8609-975c02559864 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1ba1b7d2d2a476bb12ea1266ac98afc', 'request_id': 'req-3e07f4dd-7c1f-427d-8609-975c02559864'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:16.098994 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3e07f4dd-7c1f-427d-8609-975c02559864 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1ba1b7d2d2a476bb12ea1266ac98afc', 'request_id': 'req-3e07f4dd-7c1f-427d-8609-975c02559864'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:16.105722 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3e07f4dd-7c1f-427d-8609-975c02559864 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1ba1b7d2d2a476bb12ea1266ac98afc', 'request_id': 'req-3e07f4dd-7c1f-427d-8609-975c02559864'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:16.116833 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3e07f4dd-7c1f-427d-8609-975c02559864 admin admin] The token cache is being invalidate because project a1ba1b7d2d2a476bb12ea1266ac98afc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:16.117124 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3e07f4dd-7c1f-427d-8609-975c02559864 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:16.117772 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3e07f4dd-7c1f-427d-8609-975c02559864 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:16.119547 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1015/2024] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:15 2022] DELETE /identity/v3/projects/a1ba1b7d2d2a476bb12ea1266ac98afc => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:16.124017 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42876949-ce21-448f-b599-02bd22d9b424 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:16.124425 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42876949-ce21-448f-b599-02bd22d9b424 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:16.124572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42876949-ce21-448f-b599-02bd22d9b424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:16.144063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-42876949-ce21-448f-b599-02bd22d9b424 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:16.145882 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-42876949-ce21-448f-b599-02bd22d9b424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:16.210490 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1010/2025] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:16.218088 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dd7dc-bf1d-4174-ad44-f45f623ce66d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:16.218385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dd7dc-bf1d-4174-ad44-f45f623ce66d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:16.218699 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dd7dc-bf1d-4174-ad44-f45f623ce66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:16.237159 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2b1dd7dc-bf1d-4174-ad44-f45f623ce66d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:16.238349 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2b1dd7dc-bf1d-4174-ad44-f45f623ce66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:16.246090 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1016/2026] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:16.250178 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42876949-ce21-448f-b599-02bd22d9b424 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:16.260019 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42876949-ce21-448f-b599-02bd22d9b424 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:16.264464 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41643863-12d2-411e-84fe-5a4062df8fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:16.265112 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41643863-12d2-411e-84fe-5a4062df8fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:16.265392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41643863-12d2-411e-84fe-5a4062df8fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:16.265667 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41643863-12d2-411e-84fe-5a4062df8fc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:16.266388 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41643863-12d2-411e-84fe-5a4062df8fc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:34:16.266725 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41643863-12d2-411e-84fe-5a4062df8fc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:16.268902 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41643863-12d2-411e-84fe-5a4062df8fc3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:16.280995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1011/2027] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:34:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:34:16.285838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1dd7dc-bf1d-4174-ad44-f45f623ce66d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:16.294885 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1dd7dc-bf1d-4174-ad44-f45f623ce66d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:16.299410 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4adf87-b712-442d-8ce9-8606cd722e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:16.300214 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4adf87-b712-442d-8ce9-8606cd722e39 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:16.300520 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4adf87-b712-442d-8ce9-8606cd722e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:16.300797 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4adf87-b712-442d-8ce9-8606cd722e39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:16.301518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4adf87-b712-442d-8ce9-8606cd722e39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:16.303450 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4adf87-b712-442d-8ce9-8606cd722e39 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:16.332003 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1017/2028] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:34:16 2022] POST /identity/v3/projects => generated 397 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:16.333132 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41643863-12d2-411e-84fe-5a4062df8fc3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:16.343067 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41643863-12d2-411e-84fe-5a4062df8fc3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:16.401851 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b3e67-08cb-4dce-b41a-fd0b259408ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:16.402549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b3e67-08cb-4dce-b41a-fd0b259408ef admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:16.402850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b3e67-08cb-4dce-b41a-fd0b259408ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:16.403118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b3e67-08cb-4dce-b41a-fd0b259408ef admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:16.403799 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b3e67-08cb-4dce-b41a-fd0b259408ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:16.405715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b3e67-08cb-4dce-b41a-fd0b259408ef admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:16.424773 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1012/2029] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:34:16 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:16.428843 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4adf87-b712-442d-8ce9-8606cd722e39 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:16.439195 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4adf87-b712-442d-8ce9-8606cd722e39 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:16.444455 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2b3d4-d265-43ac-ab91-6c9a584f7c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:16.445122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2b3d4-d265-43ac-ab91-6c9a584f7c68 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:16.445404 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2b3d4-d265-43ac-ab91-6c9a584f7c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:16.445678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2b3d4-d265-43ac-ab91-6c9a584f7c68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:16.446397 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea2b3d4-d265-43ac-ab91-6c9a584f7c68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:16.448006 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea2b3d4-d265-43ac-ab91-6c9a584f7c68 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:16.460084 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1018/2030] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:16.464001 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b3e67-08cb-4dce-b41a-fd0b259408ef admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:16.475136 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b3e67-08cb-4dce-b41a-fd0b259408ef admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:16.480100 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a49be-e001-4ac5-8e93-6b425d0ba89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:16.480778 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a49be-e001-4ac5-8e93-6b425d0ba89b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:16.481069 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a49be-e001-4ac5-8e93-6b425d0ba89b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:16.481398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a49be-e001-4ac5-8e93-6b425d0ba89b admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:16.482137 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336a49be-e001-4ac5-8e93-6b425d0ba89b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:16.483773 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336a49be-e001-4ac5-8e93-6b425d0ba89b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:16.499330 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1013/2031] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:34:16.500642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2b3d4-d265-43ac-ab91-6c9a584f7c68 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:16.511436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2b3d4-d265-43ac-ab91-6c9a584f7c68 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:16.516155 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5c5d2-5979-4ebb-9d33-f8afa9ccf9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:16.517046 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5c5d2-5979-4ebb-9d33-f8afa9ccf9c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:16.517338 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5c5d2-5979-4ebb-9d33-f8afa9ccf9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:16.517734 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5c5d2-5979-4ebb-9d33-f8afa9ccf9c8 admin admin] PATH_INFO: `/v3/projects/700dfa4d0dcd45a9a7f105dc7911c6d7/users/3558940234d84308834772e38b55abdc/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:16.533505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5c5d2-5979-4ebb-9d33-f8afa9ccf9c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=700dfa4d0dcd45a9a7f105dc7911c6d7, user_id=3558940234d84308834772e38b55abdc, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:16.537376 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5c5d2-5979-4ebb-9d33-f8afa9ccf9c8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:16.552439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1019/2032] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:16 2022] PUT /identity/v3/projects/700dfa4d0dcd45a9a7f105dc7911c6d7/users/3558940234d84308834772e38b55abdc/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:16.562597 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6f4f7-61ff-4fd6-bf88-889752e48f56 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:16.562894 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6f4f7-61ff-4fd6-bf88-889752e48f56 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:16.563178 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d6f4f7-61ff-4fd6-bf88-889752e48f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:16.593529 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-45d6f4f7-61ff-4fd6-bf88-889752e48f56 None None] MFA Rules not processed for user `3558940234d84308834772e38b55abdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:16.594750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-45d6f4f7-61ff-4fd6-bf88-889752e48f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:16.610152 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:16.669964 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1014/2033] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:16.674063 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5c5d2-5979-4ebb-9d33-f8afa9ccf9c8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:16.696213 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:16.740605 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5c5d2-5979-4ebb-9d33-f8afa9ccf9c8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:16.745254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54164e4f-5e92-42c5-9dce-7a5b00ed05fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:16.745920 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54164e4f-5e92-42c5-9dce-7a5b00ed05fa admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:16.746209 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54164e4f-5e92-42c5-9dce-7a5b00ed05fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:16.746525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54164e4f-5e92-42c5-9dce-7a5b00ed05fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:16.747226 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54164e4f-5e92-42c5-9dce-7a5b00ed05fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:16.749134 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54164e4f-5e92-42c5-9dce-7a5b00ed05fa admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:16.777882 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1020/2034] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:34:16 2022] POST /identity/v3/projects => generated 401 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:34:16.782200 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d6f4f7-61ff-4fd6-bf88-889752e48f56 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:16.794817 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d6f4f7-61ff-4fd6-bf88-889752e48f56 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:16.850200 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ba297-53ea-4ee6-8e74-283f8345360d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:16.850913 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ba297-53ea-4ee6-8e74-283f8345360d admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:16.851182 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ba297-53ea-4ee6-8e74-283f8345360d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:16.851462 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ba297-53ea-4ee6-8e74-283f8345360d admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:16.852145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ba297-53ea-4ee6-8e74-283f8345360d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:16.854069 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ba297-53ea-4ee6-8e74-283f8345360d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:16.878978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1015/2035] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:34:16 2022] POST /identity/v3/users => generated 307 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:16.880337 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54164e4f-5e92-42c5-9dce-7a5b00ed05fa admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:16.890590 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54164e4f-5e92-42c5-9dce-7a5b00ed05fa admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:16.895205 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f110d23-35ab-497b-81be-1268e28db138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:16.896122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f110d23-35ab-497b-81be-1268e28db138 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:16.896409 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f110d23-35ab-497b-81be-1268e28db138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:16.896681 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f110d23-35ab-497b-81be-1268e28db138 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:16.897435 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f110d23-35ab-497b-81be-1268e28db138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:16.899047 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f110d23-35ab-497b-81be-1268e28db138 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:16.910456 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1021/2036] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:34:16.914761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ba297-53ea-4ee6-8e74-283f8345360d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:16.925673 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ba297-53ea-4ee6-8e74-283f8345360d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:16.930469 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f22a13-3321-4a53-a7b7-a26ac268c97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:16.931358 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f22a13-3321-4a53-a7b7-a26ac268c97b admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:16.931646 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f22a13-3321-4a53-a7b7-a26ac268c97b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:16.931928 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f22a13-3321-4a53-a7b7-a26ac268c97b admin admin] PATH_INFO: `/v3/projects/29ada11e45a84917ad32b83f1ee6e9bc/users/27c72c2751634ffe93e790cc06ba2b15/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:16.948339 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f22a13-3321-4a53-a7b7-a26ac268c97b admin admin] RBAC: Authorizing `identity:create_grant(project_id=29ada11e45a84917ad32b83f1ee6e9bc, user_id=27c72c2751634ffe93e790cc06ba2b15, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:16.951015 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f22a13-3321-4a53-a7b7-a26ac268c97b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:16.964952 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1016/2037] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:16 2022] PUT /identity/v3/projects/29ada11e45a84917ad32b83f1ee6e9bc/users/27c72c2751634ffe93e790cc06ba2b15/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:16.968922 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f110d23-35ab-497b-81be-1268e28db138 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:16.979556 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f110d23-35ab-497b-81be-1268e28db138 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:17.040316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532dd82a-1fb6-4136-9303-6fd69288d10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:17.040926 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532dd82a-1fb6-4136-9303-6fd69288d10a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:17.041216 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532dd82a-1fb6-4136-9303-6fd69288d10a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:17.041496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532dd82a-1fb6-4136-9303-6fd69288d10a admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:17.042220 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532dd82a-1fb6-4136-9303-6fd69288d10a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:17.043848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532dd82a-1fb6-4136-9303-6fd69288d10a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:17.054123 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1022/2038] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:17.058256 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f22a13-3321-4a53-a7b7-a26ac268c97b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:17.069130 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f22a13-3321-4a53-a7b7-a26ac268c97b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:17.073693 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c91e1-2548-4f82-885a-f3626d6369bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:17.074718 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c91e1-2548-4f82-885a-f3626d6369bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:17.075017 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c91e1-2548-4f82-885a-f3626d6369bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:17.075277 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c91e1-2548-4f82-885a-f3626d6369bb admin admin] PATH_INFO: `/v3/projects/29ada11e45a84917ad32b83f1ee6e9bc/users/27c72c2751634ffe93e790cc06ba2b15/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:17.077319 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c91e1-2548-4f82-885a-f3626d6369bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=29ada11e45a84917ad32b83f1ee6e9bc, user_id=27c72c2751634ffe93e790cc06ba2b15, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:17.080120 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c91e1-2548-4f82-885a-f3626d6369bb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:17.090691 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1017/2039] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:17 2022] PUT /identity/v3/projects/29ada11e45a84917ad32b83f1ee6e9bc/users/27c72c2751634ffe93e790cc06ba2b15/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:17.102679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e793541-d5c8-4742-82cf-690b8d42a6f5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:17.103013 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e793541-d5c8-4742-82cf-690b8d42a6f5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:17.103296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e793541-d5c8-4742-82cf-690b8d42a6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:17.131109 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4e793541-d5c8-4742-82cf-690b8d42a6f5 None None] MFA Rules not processed for user `27c72c2751634ffe93e790cc06ba2b15`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:17.132330 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4e793541-d5c8-4742-82cf-690b8d42a6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:17.207412 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1023/2040] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:17 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:17.215485 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c91e1-2548-4f82-885a-f3626d6369bb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:17.217403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-486c91e1-2548-4f82-885a-f3626d6369bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:17.278413 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c91e1-2548-4f82-885a-f3626d6369bb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:17.282109 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5089a6-c6ec-4226-9db1-76626cb17111 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:17.282813 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5089a6-c6ec-4226-9db1-76626cb17111 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:17.283093 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5089a6-c6ec-4226-9db1-76626cb17111 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:17.283368 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5089a6-c6ec-4226-9db1-76626cb17111 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:17.294183 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5089a6-c6ec-4226-9db1-76626cb17111 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:17.295582 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5089a6-c6ec-4226-9db1-76626cb17111 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:17.306196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1018/2041] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:17 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:17.379802 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e793541-d5c8-4742-82cf-690b8d42a6f5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:17.381468 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4e793541-d5c8-4742-82cf-690b8d42a6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:17.394731 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e793541-d5c8-4742-82cf-690b8d42a6f5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:17.398964 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e18ebd-eeab-4b30-96e1-4ea1f4ed8b8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:17.399619 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e18ebd-eeab-4b30-96e1-4ea1f4ed8b8c service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:17.399947 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e18ebd-eeab-4b30-96e1-4ea1f4ed8b8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:17.400174 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e18ebd-eeab-4b30-96e1-4ea1f4ed8b8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:17.413332 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:17.461015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e18ebd-eeab-4b30-96e1-4ea1f4ed8b8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:17.462467 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e18ebd-eeab-4b30-96e1-4ea1f4ed8b8c service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:17.471553 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1024/2042] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:17 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:22.093501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5089a6-c6ec-4226-9db1-76626cb17111 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:22.095109 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1d5089a6-c6ec-4226-9db1-76626cb17111 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:22.102892 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:22.171500 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5089a6-c6ec-4226-9db1-76626cb17111 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:22.178909 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc8e7e-0913-4373-8c9d-0d281e771077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:22.179892 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc8e7e-0913-4373-8c9d-0d281e771077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:22.180181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc8e7e-0913-4373-8c9d-0d281e771077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:22.180465 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc8e7e-0913-4373-8c9d-0d281e771077 admin admin] PATH_INFO: `/v3/users/743d985dd40d4d35a3c1364ba6622524` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:22.181845 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc8e7e-0913-4373-8c9d-0d281e771077 admin admin] RBAC: Authorizing `identity:delete_user(user_id=743d985dd40d4d35a3c1364ba6622524)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:22.187745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc8e7e-0913-4373-8c9d-0d281e771077 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:22.248146 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-70bc8e7e-0913-4373-8c9d-0d281e771077 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '743d985dd40d4d35a3c1364ba6622524'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:22.264227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-70bc8e7e-0913-4373-8c9d-0d281e771077 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '743d985dd40d4d35a3c1364ba6622524'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:22.276521 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-70bc8e7e-0913-4373-8c9d-0d281e771077 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '743d985dd40d4d35a3c1364ba6622524'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:22.277170 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-70bc8e7e-0913-4373-8c9d-0d281e771077 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '743d985dd40d4d35a3c1364ba6622524'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:22.277776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-70bc8e7e-0913-4373-8c9d-0d281e771077 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '743d985dd40d4d35a3c1364ba6622524'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:22.291303 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1019/2043] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:34:22 2022] DELETE /identity/v3/users/743d985dd40d4d35a3c1364ba6622524 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:22.296080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d6ca9-255e-4579-b888-91f39865ff23 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:22.296610 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d6ca9-255e-4579-b888-91f39865ff23 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:22.296705 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5d6ca9-255e-4579-b888-91f39865ff23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:22.314930 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8b5d6ca9-255e-4579-b888-91f39865ff23 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:22.316753 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8b5d6ca9-255e-4579-b888-91f39865ff23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:22.384402 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1025/2044] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:22.396001 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc8e7e-0913-4373-8c9d-0d281e771077 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:22.399205 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-70bc8e7e-0913-4373-8c9d-0d281e771077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:22.464312 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc8e7e-0913-4373-8c9d-0d281e771077 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:22.468024 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46628c3a-a0ed-48a1-9cf6-984c2afc620f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:22.468781 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46628c3a-a0ed-48a1-9cf6-984c2afc620f service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:22.468960 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46628c3a-a0ed-48a1-9cf6-984c2afc620f service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:22.469231 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46628c3a-a0ed-48a1-9cf6-984c2afc620f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:22.480124 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46628c3a-a0ed-48a1-9cf6-984c2afc620f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:22.481733 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46628c3a-a0ed-48a1-9cf6-984c2afc620f service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:22.493600 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1020/2045] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:22.903554 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d6ca9-255e-4579-b888-91f39865ff23 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:22.905415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8b5d6ca9-255e-4579-b888-91f39865ff23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:22.919362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5d6ca9-255e-4579-b888-91f39865ff23 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:22.924972 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb49732-fd76-4266-a55a-0b94975cd5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:22.927335 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb49732-fd76-4266-a55a-0b94975cd5ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:22.927839 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb49732-fd76-4266-a55a-0b94975cd5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:22.928291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb49732-fd76-4266-a55a-0b94975cd5ff admin admin] PATH_INFO: `/v3/users/3558940234d84308834772e38b55abdc` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:22.929951 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb49732-fd76-4266-a55a-0b94975cd5ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=3558940234d84308834772e38b55abdc)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:22.932210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb49732-fd76-4266-a55a-0b94975cd5ff admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:22.995400 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46628c3a-a0ed-48a1-9cf6-984c2afc620f service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:22.997146 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-46628c3a-a0ed-48a1-9cf6-984c2afc620f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:23.017079 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-adb49732-fd76-4266-a55a-0b94975cd5ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3558940234d84308834772e38b55abdc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:23.017744 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-adb49732-fd76-4266-a55a-0b94975cd5ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3558940234d84308834772e38b55abdc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:23.028706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46628c3a-a0ed-48a1-9cf6-984c2afc620f service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:23.033005 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-adb49732-fd76-4266-a55a-0b94975cd5ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3558940234d84308834772e38b55abdc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:23.033711 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-adb49732-fd76-4266-a55a-0b94975cd5ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3558940234d84308834772e38b55abdc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:23.037763 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7313a9-00f0-411b-9a07-f03be14d8052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:23.038665 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7313a9-00f0-411b-9a07-f03be14d8052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:23.038958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7313a9-00f0-411b-9a07-f03be14d8052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:23.039371 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7313a9-00f0-411b-9a07-f03be14d8052 admin admin] PATH_INFO: `/v3/projects/91caf8abad6c46039501fd01890434d2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:23.041705 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7313a9-00f0-411b-9a07-f03be14d8052 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91caf8abad6c46039501fd01890434d2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:23.042154 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-adb49732-fd76-4266-a55a-0b94975cd5ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3558940234d84308834772e38b55abdc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:23.043796 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7313a9-00f0-411b-9a07-f03be14d8052 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:23.057428 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1026/2046] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:34:22 2022] DELETE /identity/v3/users/3558940234d84308834772e38b55abdc => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:23.061629 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee83f20-f0af-4bab-910b-5ffb38c9cf9e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:23.061925 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee83f20-f0af-4bab-910b-5ffb38c9cf9e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:23.062210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee83f20-f0af-4bab-910b-5ffb38c9cf9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:23.072374 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:23.079969 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3ee83f20-f0af-4bab-910b-5ffb38c9cf9e None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:23.081521 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3ee83f20-f0af-4bab-910b-5ffb38c9cf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:23.101123 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2b7313a9-00f0-411b-9a07-f03be14d8052 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91caf8abad6c46039501fd01890434d2', 'request_id': 'req-2b7313a9-00f0-411b-9a07-f03be14d8052'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:23.104217 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:23.108381 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2b7313a9-00f0-411b-9a07-f03be14d8052 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91caf8abad6c46039501fd01890434d2', 'request_id': 'req-2b7313a9-00f0-411b-9a07-f03be14d8052'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:23.115212 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2b7313a9-00f0-411b-9a07-f03be14d8052 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91caf8abad6c46039501fd01890434d2', 'request_id': 'req-2b7313a9-00f0-411b-9a07-f03be14d8052'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:23.128582 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2b7313a9-00f0-411b-9a07-f03be14d8052 admin admin] The token cache is being invalidate because project 91caf8abad6c46039501fd01890434d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:23.128875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2b7313a9-00f0-411b-9a07-f03be14d8052 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:23.129505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2b7313a9-00f0-411b-9a07-f03be14d8052 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:23.131162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1021/2047] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:22 2022] DELETE /identity/v3/projects/91caf8abad6c46039501fd01890434d2 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:23.136755 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf1949-a9da-4738-8c65-92d1563b7570 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:23.137045 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf1949-a9da-4738-8c65-92d1563b7570 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:23.137327 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf1949-a9da-4738-8c65-92d1563b7570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:23.146887 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1027/2048] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:34:23.155960 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d9cf1949-a9da-4738-8c65-92d1563b7570 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:23.156458 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee83f20-f0af-4bab-910b-5ffb38c9cf9e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:23.157504 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d9cf1949-a9da-4738-8c65-92d1563b7570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:23.157903 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3ee83f20-f0af-4bab-910b-5ffb38c9cf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:23.216851 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee83f20-f0af-4bab-910b-5ffb38c9cf9e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:23.225052 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1022/2049] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:34:23.232840 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d436c1-b284-4b63-8c7e-927bb27ec705 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:23.233135 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d436c1-b284-4b63-8c7e-927bb27ec705 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:23.233596 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d436c1-b284-4b63-8c7e-927bb27ec705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:23.254186 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-30d436c1-b284-4b63-8c7e-927bb27ec705 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:23.256131 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-30d436c1-b284-4b63-8c7e-927bb27ec705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:23.264444 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1023/2050] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:34:23.269434 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d436c1-b284-4b63-8c7e-927bb27ec705 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:23.277194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ecf24-2cdf-40e3-b2df-2a319b3a9225 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:23.278014 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ecf24-2cdf-40e3-b2df-2a319b3a9225 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:23.278270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ecf24-2cdf-40e3-b2df-2a319b3a9225 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:23.278580 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ecf24-2cdf-40e3-b2df-2a319b3a9225 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:23.278766 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d436c1-b284-4b63-8c7e-927bb27ec705 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:23.284426 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b8481-fbbd-46c7-bd69-4c13d6ee5869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:23.285099 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b8481-fbbd-46c7-bd69-4c13d6ee5869 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:23.285384 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b8481-fbbd-46c7-bd69-4c13d6ee5869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:23.285681 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b8481-fbbd-46c7-bd69-4c13d6ee5869 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:23.286392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b8481-fbbd-46c7-bd69-4c13d6ee5869 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:34:23.286735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b8481-fbbd-46c7-bd69-4c13d6ee5869 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:23.288352 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b8481-fbbd-46c7-bd69-4c13d6ee5869 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:23.291123 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ecf24-2cdf-40e3-b2df-2a319b3a9225 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:23.292550 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ecf24-2cdf-40e3-b2df-2a319b3a9225 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:23.302826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1024/2051] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:34:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:34:23.307184 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b8481-fbbd-46c7-bd69-4c13d6ee5869 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:23.307813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1028/2052] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 12:34:23.318778 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b8481-fbbd-46c7-bd69-4c13d6ee5869 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:23.323801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278052be-4e03-475e-b1bd-b2b977b1a6e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:23.324298 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278052be-4e03-475e-b1bd-b2b977b1a6e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:23.324591 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278052be-4e03-475e-b1bd-b2b977b1a6e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:23.324873 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278052be-4e03-475e-b1bd-b2b977b1a6e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:23.325571 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278052be-4e03-475e-b1bd-b2b977b1a6e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:23.327550 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278052be-4e03-475e-b1bd-b2b977b1a6e5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:23.373226 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1025/2053] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:34:23 2022] POST /identity/v3/projects => generated 369 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:23.377207 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ecf24-2cdf-40e3-b2df-2a319b3a9225 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:23.388210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ecf24-2cdf-40e3-b2df-2a319b3a9225 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:23.473066 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9094d851-ad24-414c-8a54-0f32f70333ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:23.474009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094d851-ad24-414c-8a54-0f32f70333ee admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:23.474517 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094d851-ad24-414c-8a54-0f32f70333ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:23.474805 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094d851-ad24-414c-8a54-0f32f70333ee admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:23.475529 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9094d851-ad24-414c-8a54-0f32f70333ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:23.491963 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9094d851-ad24-414c-8a54-0f32f70333ee admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:23.515770 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1029/2054] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:34:23 2022] POST /identity/v3/users => generated 285 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:23.521240 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278052be-4e03-475e-b1bd-b2b977b1a6e5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:23.533629 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278052be-4e03-475e-b1bd-b2b977b1a6e5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:23.539672 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf21f5a-831e-4208-8bab-124782dd0579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:23.540571 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf21f5a-831e-4208-8bab-124782dd0579 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:23.540964 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf21f5a-831e-4208-8bab-124782dd0579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:23.541317 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf21f5a-831e-4208-8bab-124782dd0579 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:23.542366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf21f5a-831e-4208-8bab-124782dd0579 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:23.543926 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf21f5a-831e-4208-8bab-124782dd0579 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:23.558079 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1026/2055] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:23.559344 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9094d851-ad24-414c-8a54-0f32f70333ee admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:23.579703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9094d851-ad24-414c-8a54-0f32f70333ee admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:23.585812 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b9679-d454-4f9f-beb2-45c7606e526a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:23.586532 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b9679-d454-4f9f-beb2-45c7606e526a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:23.586820 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b9679-d454-4f9f-beb2-45c7606e526a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:23.587124 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b9679-d454-4f9f-beb2-45c7606e526a admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:23.587860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b9679-d454-4f9f-beb2-45c7606e526a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:23.589499 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b9679-d454-4f9f-beb2-45c7606e526a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:23.603348 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1030/2056] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:23 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:34:23.607675 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf21f5a-831e-4208-8bab-124782dd0579 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:23.619982 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf21f5a-831e-4208-8bab-124782dd0579 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:23.625084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451617f2-779d-4020-880a-bda14304253b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:23.626335 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451617f2-779d-4020-880a-bda14304253b admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:23.626816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451617f2-779d-4020-880a-bda14304253b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:23.627311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451617f2-779d-4020-880a-bda14304253b admin admin] PATH_INFO: `/v3/projects/aebcb8dc38ef49da8f70ec08df95e12b/users/99d1bd1790c049e5869f61cfd7b51d25/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:23.631709 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b9679-d454-4f9f-beb2-45c7606e526a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:23.633500 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bf4b9679-d454-4f9f-beb2-45c7606e526a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:23.643675 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:23.648757 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b9679-d454-4f9f-beb2-45c7606e526a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:23.659581 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985c923-1439-4f9e-8249-f4f64e12138d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:23.660447 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985c923-1439-4f9e-8249-f4f64e12138d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:23.660768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985c923-1439-4f9e-8249-f4f64e12138d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:23.661060 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985c923-1439-4f9e-8249-f4f64e12138d admin admin] PATH_INFO: `/v3/projects/700dfa4d0dcd45a9a7f105dc7911c6d7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:23.662747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f985c923-1439-4f9e-8249-f4f64e12138d admin admin] RBAC: Authorizing `identity:delete_project(project_id=700dfa4d0dcd45a9a7f105dc7911c6d7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:23.665111 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f985c923-1439-4f9e-8249-f4f64e12138d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:23.670675 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451617f2-779d-4020-880a-bda14304253b admin admin] RBAC: Authorizing `identity:create_grant(project_id=aebcb8dc38ef49da8f70ec08df95e12b, user_id=99d1bd1790c049e5869f61cfd7b51d25, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:23.674935 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451617f2-779d-4020-880a-bda14304253b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:23.696901 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1027/2057] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:23 2022] PUT /identity/v3/projects/aebcb8dc38ef49da8f70ec08df95e12b/users/99d1bd1790c049e5869f61cfd7b51d25/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:23.707665 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4f304-af22-49ec-a59e-1d6eed5c5325 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:23.707958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4f304-af22-49ec-a59e-1d6eed5c5325 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:23.708246 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4f304-af22-49ec-a59e-1d6eed5c5325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:23.735733 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3aa4f304-af22-49ec-a59e-1d6eed5c5325 None None] MFA Rules not processed for user `99d1bd1790c049e5869f61cfd7b51d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:23.737485 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3aa4f304-af22-49ec-a59e-1d6eed5c5325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:23.739083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f985c923-1439-4f9e-8249-f4f64e12138d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '700dfa4d0dcd45a9a7f105dc7911c6d7', 'request_id': 'req-f985c923-1439-4f9e-8249-f4f64e12138d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:23.749327 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f985c923-1439-4f9e-8249-f4f64e12138d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '700dfa4d0dcd45a9a7f105dc7911c6d7', 'request_id': 'req-f985c923-1439-4f9e-8249-f4f64e12138d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:23.760086 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f985c923-1439-4f9e-8249-f4f64e12138d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '700dfa4d0dcd45a9a7f105dc7911c6d7', 'request_id': 'req-f985c923-1439-4f9e-8249-f4f64e12138d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:23.764189 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:23.774951 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f985c923-1439-4f9e-8249-f4f64e12138d admin admin] The token cache is being invalidate because project 700dfa4d0dcd45a9a7f105dc7911c6d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:23.775410 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f985c923-1439-4f9e-8249-f4f64e12138d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:23.776489 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f985c923-1439-4f9e-8249-f4f64e12138d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:23.778273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1031/2058] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:23 2022] DELETE /identity/v3/projects/700dfa4d0dcd45a9a7f105dc7911c6d7 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:23.782303 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985c923-1439-4f9e-8249-f4f64e12138d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:23.784238 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f985c923-1439-4f9e-8249-f4f64e12138d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:23.826870 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1028/2059] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:23 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:23.840960 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa4f304-af22-49ec-a59e-1d6eed5c5325 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:23.842796 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3aa4f304-af22-49ec-a59e-1d6eed5c5325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:23.855456 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f985c923-1439-4f9e-8249-f4f64e12138d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:23.860309 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a127f-6472-405d-bbef-75b7879f4c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:23.861193 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a127f-6472-405d-bbef-75b7879f4c76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:23.861483 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a127f-6472-405d-bbef-75b7879f4c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:23.861766 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a127f-6472-405d-bbef-75b7879f4c76 admin admin] PATH_INFO: `/v3/users/27c72c2751634ffe93e790cc06ba2b15` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:23.863449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97a127f-6472-405d-bbef-75b7879f4c76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27c72c2751634ffe93e790cc06ba2b15)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:23.865380 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97a127f-6472-405d-bbef-75b7879f4c76 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:23.907268 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa4f304-af22-49ec-a59e-1d6eed5c5325 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:23.911147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f9ac0-e5b6-4190-ac74-01818c64907a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '096234e9f15541c0b187dc8dd527b0fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:23.911797 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f9ac0-e5b6-4190-ac74-01818c64907a service glance] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:23.912087 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f9ac0-e5b6-4190-ac74-01818c64907a service glance] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:23.912364 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f9ac0-e5b6-4190-ac74-01818c64907a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:23.916296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d97a127f-6472-405d-bbef-75b7879f4c76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27c72c2751634ffe93e790cc06ba2b15'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:23.916961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d97a127f-6472-405d-bbef-75b7879f4c76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27c72c2751634ffe93e790cc06ba2b15'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:23.922203 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f9ac0-e5b6-4190-ac74-01818c64907a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:23.923893 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f9ac0-e5b6-4190-ac74-01818c64907a service glance] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:23.929674 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d97a127f-6472-405d-bbef-75b7879f4c76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27c72c2751634ffe93e790cc06ba2b15'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:23.930365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d97a127f-6472-405d-bbef-75b7879f4c76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27c72c2751634ffe93e790cc06ba2b15'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:23.933201 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1029/2060] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:23 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 12:34:23.938898 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d97a127f-6472-405d-bbef-75b7879f4c76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27c72c2751634ffe93e790cc06ba2b15'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:23.953368 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1032/2061] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:34:23 2022] DELETE /identity/v3/users/27c72c2751634ffe93e790cc06ba2b15 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:34:24.304401 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f9ac0-e5b6-4190-ac74-01818c64907a service glance] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:24.325088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f9ac0-e5b6-4190-ac74-01818c64907a service glance] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:24.413879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a171d9-4691-478d-b29c-0e7e96902dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:24.414882 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a171d9-4691-478d-b29c-0e7e96902dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:24.415173 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a171d9-4691-478d-b29c-0e7e96902dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:24.415509 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a171d9-4691-478d-b29c-0e7e96902dd5 admin admin] PATH_INFO: `/v3/projects/29ada11e45a84917ad32b83f1ee6e9bc` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:24.417893 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a171d9-4691-478d-b29c-0e7e96902dd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29ada11e45a84917ad32b83f1ee6e9bc)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:24.418638 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a171d9-4691-478d-b29c-0e7e96902dd5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:24.468410 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-42a171d9-4691-478d-b29c-0e7e96902dd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29ada11e45a84917ad32b83f1ee6e9bc', 'request_id': 'req-42a171d9-4691-478d-b29c-0e7e96902dd5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:24.475178 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-42a171d9-4691-478d-b29c-0e7e96902dd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29ada11e45a84917ad32b83f1ee6e9bc', 'request_id': 'req-42a171d9-4691-478d-b29c-0e7e96902dd5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:24.480633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-42a171d9-4691-478d-b29c-0e7e96902dd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29ada11e45a84917ad32b83f1ee6e9bc', 'request_id': 'req-42a171d9-4691-478d-b29c-0e7e96902dd5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:24.484117 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:24.495326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-42a171d9-4691-478d-b29c-0e7e96902dd5 admin admin] The token cache is being invalidate because project 29ada11e45a84917ad32b83f1ee6e9bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:24.495655 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-42a171d9-4691-478d-b29c-0e7e96902dd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:24.496675 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-42a171d9-4691-478d-b29c-0e7e96902dd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:24.498414 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1030/2062] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:24 2022] DELETE /identity/v3/projects/29ada11e45a84917ad32b83f1ee6e9bc => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:24.503573 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56371f-4c59-4f23-b626-ac42933753dc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:24.504028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56371f-4c59-4f23-b626-ac42933753dc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:24.504277 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff56371f-4c59-4f23-b626-ac42933753dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:24.516740 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:24.524329 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ff56371f-4c59-4f23-b626-ac42933753dc None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:24.526223 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ff56371f-4c59-4f23-b626-ac42933753dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:24.588089 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1033/2063] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:24.595805 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76891d14-f13a-4c6e-ba7f-992b582aaccd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:24.596122 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76891d14-f13a-4c6e-ba7f-992b582aaccd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:24.596578 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76891d14-f13a-4c6e-ba7f-992b582aaccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:24.616236 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-76891d14-f13a-4c6e-ba7f-992b582aaccd None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:24.617441 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-76891d14-f13a-4c6e-ba7f-992b582aaccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:24.625356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1031/2064] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:24.630108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56371f-4c59-4f23-b626-ac42933753dc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:24.639859 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff56371f-4c59-4f23-b626-ac42933753dc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:24.644647 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0610f0d8-ee40-42fd-bf98-ec1ab975ae18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:24.645551 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610f0d8-ee40-42fd-bf98-ec1ab975ae18 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:24.645840 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610f0d8-ee40-42fd-bf98-ec1ab975ae18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:24.646123 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610f0d8-ee40-42fd-bf98-ec1ab975ae18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:24.646890 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0610f0d8-ee40-42fd-bf98-ec1ab975ae18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:34:24.647233 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0610f0d8-ee40-42fd-bf98-ec1ab975ae18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:24.648844 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0610f0d8-ee40-42fd-bf98-ec1ab975ae18 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:24.660930 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1034/2065] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:34:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:34:24.665002 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76891d14-f13a-4c6e-ba7f-992b582aaccd None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:24.674173 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76891d14-f13a-4c6e-ba7f-992b582aaccd None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:24.679541 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12c655-e950-426e-86ea-0ffad5f7cf96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:24.680185 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12c655-e950-426e-86ea-0ffad5f7cf96 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:24.682991 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12c655-e950-426e-86ea-0ffad5f7cf96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:24.683358 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d12c655-e950-426e-86ea-0ffad5f7cf96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:24.684248 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12c655-e950-426e-86ea-0ffad5f7cf96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:24.686587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d12c655-e950-426e-86ea-0ffad5f7cf96 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:24.713871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1032/2066] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:34:24 2022] POST /identity/v3/projects => generated 409 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:34:24.717943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0610f0d8-ee40-42fd-bf98-ec1ab975ae18 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:24.728644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0610f0d8-ee40-42fd-bf98-ec1ab975ae18 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:24.795807 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6b339-339b-400b-8d7b-72529a6b439b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:24.796490 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6b339-339b-400b-8d7b-72529a6b439b admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:24.796779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6b339-339b-400b-8d7b-72529a6b439b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:24.797060 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6b339-339b-400b-8d7b-72529a6b439b admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:24.797773 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6b339-339b-400b-8d7b-72529a6b439b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:24.800840 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6b339-339b-400b-8d7b-72529a6b439b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:24.823904 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1035/2067] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:34:24 2022] POST /identity/v3/users => generated 305 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:24.825086 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12c655-e950-426e-86ea-0ffad5f7cf96 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:24.836217 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d12c655-e950-426e-86ea-0ffad5f7cf96 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:24.841010 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc58cc-dfc2-469d-ac09-6cbd2c5d0a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:24.841679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc58cc-dfc2-469d-ac09-6cbd2c5d0a2a admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:24.841966 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc58cc-dfc2-469d-ac09-6cbd2c5d0a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:24.842248 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc58cc-dfc2-469d-ac09-6cbd2c5d0a2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:24.843031 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc58cc-dfc2-469d-ac09-6cbd2c5d0a2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:24.844640 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc58cc-dfc2-469d-ac09-6cbd2c5d0a2a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:24.856491 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1033/2068] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:34:24.860459 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6b339-339b-400b-8d7b-72529a6b439b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:24.871562 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6b339-339b-400b-8d7b-72529a6b439b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:24.876430 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe53720-d025-4287-98c1-b55b357c67f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:24.877085 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe53720-d025-4287-98c1-b55b357c67f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:24.877370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe53720-d025-4287-98c1-b55b357c67f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:24.877745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe53720-d025-4287-98c1-b55b357c67f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:24.878590 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe53720-d025-4287-98c1-b55b357c67f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:24.880478 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe53720-d025-4287-98c1-b55b357c67f6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:24.894302 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1036/2069] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:24.895819 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc58cc-dfc2-469d-ac09-6cbd2c5d0a2a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:24.906468 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc58cc-dfc2-469d-ac09-6cbd2c5d0a2a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:24.913929 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b26b7-b6ca-4259-99fe-99b10c7c7a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:24.916629 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b26b7-b6ca-4259-99fe-99b10c7c7a6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:24.916820 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b26b7-b6ca-4259-99fe-99b10c7c7a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:24.917057 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b26b7-b6ca-4259-99fe-99b10c7c7a6e admin admin] PATH_INFO: `/v3/projects/7c45bf36ed544f0a9f5ee87df819c7d0/users/0377aca6d6334cb2a764eb583618da98/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:24.934076 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4b26b7-b6ca-4259-99fe-99b10c7c7a6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c45bf36ed544f0a9f5ee87df819c7d0, user_id=0377aca6d6334cb2a764eb583618da98, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:24.937146 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4b26b7-b6ca-4259-99fe-99b10c7c7a6e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:24.951002 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1034/2070] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:24 2022] PUT /identity/v3/projects/7c45bf36ed544f0a9f5ee87df819c7d0/users/0377aca6d6334cb2a764eb583618da98/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:24.964169 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47679196-7c9f-4da4-b2b8-ee43d14d8036 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:24.964465 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47679196-7c9f-4da4-b2b8-ee43d14d8036 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:24.964748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47679196-7c9f-4da4-b2b8-ee43d14d8036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:24.995627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-47679196-7c9f-4da4-b2b8-ee43d14d8036 None None] MFA Rules not processed for user `0377aca6d6334cb2a764eb583618da98`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:24.996954 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-47679196-7c9f-4da4-b2b8-ee43d14d8036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:25.068706 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1037/2071] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:24 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:25.075047 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b26b7-b6ca-4259-99fe-99b10c7c7a6e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:25.153964 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b26b7-b6ca-4259-99fe-99b10c7c7a6e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:25.165066 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d4e3c-13bc-4fa2-8a7e-5a24b0e157e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:25.166206 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d4e3c-13bc-4fa2-8a7e-5a24b0e157e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:25.166699 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d4e3c-13bc-4fa2-8a7e-5a24b0e157e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:25.167108 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d4e3c-13bc-4fa2-8a7e-5a24b0e157e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:25.167839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d4e3c-13bc-4fa2-8a7e-5a24b0e157e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:25.169932 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071d4e3c-13bc-4fa2-8a7e-5a24b0e157e6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:25.206810 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1035/2072] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:34:25 2022] POST /identity/v3/projects => generated 407 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:25.211350 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47679196-7c9f-4da4-b2b8-ee43d14d8036 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:25.221285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:25.231871 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47679196-7c9f-4da4-b2b8-ee43d14d8036 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:25.299525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9f2e2-24d8-4d61-b2be-b4a1fa9b4cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:25.300828 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9f2e2-24d8-4d61-b2be-b4a1fa9b4cfc admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:25.301294 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9f2e2-24d8-4d61-b2be-b4a1fa9b4cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:25.301770 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9f2e2-24d8-4d61-b2be-b4a1fa9b4cfc admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:25.302699 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb9f2e2-24d8-4d61-b2be-b4a1fa9b4cfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:25.304979 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb9f2e2-24d8-4d61-b2be-b4a1fa9b4cfc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:25.337016 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1038/2073] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:34:25 2022] POST /identity/v3/users => generated 310 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:34:25.341600 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d4e3c-13bc-4fa2-8a7e-5a24b0e157e6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:25.356497 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d4e3c-13bc-4fa2-8a7e-5a24b0e157e6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:25.376317 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e05152-582d-486d-9360-7765dd56aa86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:25.380166 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e05152-582d-486d-9360-7765dd56aa86 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:25.380564 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e05152-582d-486d-9360-7765dd56aa86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:25.380910 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e05152-582d-486d-9360-7765dd56aa86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:25.381773 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e05152-582d-486d-9360-7765dd56aa86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:25.387805 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e05152-582d-486d-9360-7765dd56aa86 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:25.413455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1036/2074] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:25.414661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9f2e2-24d8-4d61-b2be-b4a1fa9b4cfc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:25.429586 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9f2e2-24d8-4d61-b2be-b4a1fa9b4cfc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:25.441923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca6d65-5914-4f5a-a333-5f1539cf03e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:25.443258 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca6d65-5914-4f5a-a333-5f1539cf03e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:25.443976 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca6d65-5914-4f5a-a333-5f1539cf03e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:25.444615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca6d65-5914-4f5a-a333-5f1539cf03e8 admin admin] PATH_INFO: `/v3/projects/053e29d4ac0f47e8956d3f5d7a1b4598/users/867fedeb546449fdaa9292e39befaa49/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:25.464808 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca6d65-5914-4f5a-a333-5f1539cf03e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=053e29d4ac0f47e8956d3f5d7a1b4598, user_id=867fedeb546449fdaa9292e39befaa49, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:25.468472 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca6d65-5914-4f5a-a333-5f1539cf03e8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:25.485012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1039/2075] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:25 2022] PUT /identity/v3/projects/053e29d4ac0f47e8956d3f5d7a1b4598/users/867fedeb546449fdaa9292e39befaa49/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:25.489176 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e05152-582d-486d-9360-7765dd56aa86 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:25.499699 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e05152-582d-486d-9360-7765dd56aa86 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:25.561198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ccf949-6b65-466e-97f0-617b4681d651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:25.561875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ccf949-6b65-466e-97f0-617b4681d651 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:25.562159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ccf949-6b65-466e-97f0-617b4681d651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:25.562470 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ccf949-6b65-466e-97f0-617b4681d651 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:25.563198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ccf949-6b65-466e-97f0-617b4681d651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:25.564850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ccf949-6b65-466e-97f0-617b4681d651 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:25.577021 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1037/2076] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:25.582155 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca6d65-5914-4f5a-a333-5f1539cf03e8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:25.595487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca6d65-5914-4f5a-a333-5f1539cf03e8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:25.600955 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e68ef6-dc38-4491-9552-6ed51e232283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:25.601854 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e68ef6-dc38-4491-9552-6ed51e232283 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:25.602143 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e68ef6-dc38-4491-9552-6ed51e232283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:25.602507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e68ef6-dc38-4491-9552-6ed51e232283 admin admin] PATH_INFO: `/v3/projects/053e29d4ac0f47e8956d3f5d7a1b4598/users/867fedeb546449fdaa9292e39befaa49/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:25.604587 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e68ef6-dc38-4491-9552-6ed51e232283 admin admin] RBAC: Authorizing `identity:create_grant(project_id=053e29d4ac0f47e8956d3f5d7a1b4598, user_id=867fedeb546449fdaa9292e39befaa49, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:25.607265 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e68ef6-dc38-4491-9552-6ed51e232283 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:25.618788 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1040/2077] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:25 2022] PUT /identity/v3/projects/053e29d4ac0f47e8956d3f5d7a1b4598/users/867fedeb546449fdaa9292e39befaa49/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:34:25.629407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d169b81-5ea8-484b-8ea3-4df1af732e2e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:25.629695 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d169b81-5ea8-484b-8ea3-4df1af732e2e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:25.629973 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d169b81-5ea8-484b-8ea3-4df1af732e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:25.636143 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:25.659428 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1d169b81-5ea8-484b-8ea3-4df1af732e2e None None] MFA Rules not processed for user `867fedeb546449fdaa9292e39befaa49`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:25.660642 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1d169b81-5ea8-484b-8ea3-4df1af732e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:25.731239 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1038/2078] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:25 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:25.742650 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e68ef6-dc38-4491-9552-6ed51e232283 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:25.744367 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f3e68ef6-dc38-4491-9552-6ed51e232283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:25.803979 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e68ef6-dc38-4491-9552-6ed51e232283 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:25.807505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f403655-7d6e-4664-83f1-c72737154990 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:25.808150 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f403655-7d6e-4664-83f1-c72737154990 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:25.808434 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f403655-7d6e-4664-83f1-c72737154990 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:25.808710 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f403655-7d6e-4664-83f1-c72737154990 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:25.818323 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f403655-7d6e-4664-83f1-c72737154990 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:25.819842 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f403655-7d6e-4664-83f1-c72737154990 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:25.829228 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1041/2079] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:25 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 12:34:26.005566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d169b81-5ea8-484b-8ea3-4df1af732e2e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:26.018056 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d169b81-5ea8-484b-8ea3-4df1af732e2e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:26.022094 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e080a2-ba84-475a-bf6f-0449c23e698f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:26.022806 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e080a2-ba84-475a-bf6f-0449c23e698f service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:26.023094 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e080a2-ba84-475a-bf6f-0449c23e698f service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:26.023367 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e080a2-ba84-475a-bf6f-0449c23e698f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:26.093529 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e080a2-ba84-475a-bf6f-0449c23e698f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:26.096926 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e080a2-ba84-475a-bf6f-0449c23e698f service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:26.111882 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1039/2080] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:26 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:34.785379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f403655-7d6e-4664-83f1-c72737154990 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:34.786861 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1f403655-7d6e-4664-83f1-c72737154990 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:34.894846 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:34.901630 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f403655-7d6e-4664-83f1-c72737154990 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:34.907642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261a437-1134-4a3f-8ff9-1ef36131611c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:34.908569 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261a437-1134-4a3f-8ff9-1ef36131611c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:34.909137 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261a437-1134-4a3f-8ff9-1ef36131611c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:34.909587 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261a437-1134-4a3f-8ff9-1ef36131611c admin admin] PATH_INFO: `/v3/users/aa09e7713f534793ab380c309ed817b7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:34.910853 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2261a437-1134-4a3f-8ff9-1ef36131611c admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa09e7713f534793ab380c309ed817b7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:34.913135 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2261a437-1134-4a3f-8ff9-1ef36131611c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:34.996067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2261a437-1134-4a3f-8ff9-1ef36131611c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa09e7713f534793ab380c309ed817b7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:34.997160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2261a437-1134-4a3f-8ff9-1ef36131611c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa09e7713f534793ab380c309ed817b7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:35.014449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2261a437-1134-4a3f-8ff9-1ef36131611c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa09e7713f534793ab380c309ed817b7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:35.015455 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2261a437-1134-4a3f-8ff9-1ef36131611c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa09e7713f534793ab380c309ed817b7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:35.025957 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2261a437-1134-4a3f-8ff9-1ef36131611c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa09e7713f534793ab380c309ed817b7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:35.041668 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1042/2081] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:34:34 2022] DELETE /identity/v3/users/aa09e7713f534793ab380c309ed817b7 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:35.046533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5a704-94fc-4857-bd47-f5ba5849721d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:35.047047 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5a704-94fc-4857-bd47-f5ba5849721d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:35.047218 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5a704-94fc-4857-bd47-f5ba5849721d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:35.066629 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-78e5a704-94fc-4857-bd47-f5ba5849721d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:35.068148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-78e5a704-94fc-4857-bd47-f5ba5849721d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:35.133902 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1040/2082] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:35.146065 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261a437-1134-4a3f-8ff9-1ef36131611c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:35.148057 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2261a437-1134-4a3f-8ff9-1ef36131611c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:35.216615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261a437-1134-4a3f-8ff9-1ef36131611c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:35.220356 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a43ece-7a51-45d9-9d07-5bff7d4a89eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:35.221616 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a43ece-7a51-45d9-9d07-5bff7d4a89eb service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:35.222573 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a43ece-7a51-45d9-9d07-5bff7d4a89eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:35.222861 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a43ece-7a51-45d9-9d07-5bff7d4a89eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:35.234015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a43ece-7a51-45d9-9d07-5bff7d4a89eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:35.235799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a43ece-7a51-45d9-9d07-5bff7d4a89eb service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:35.247236 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1043/2083] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:35.556525 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5a704-94fc-4857-bd47-f5ba5849721d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:35.558224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-78e5a704-94fc-4857-bd47-f5ba5849721d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:35.571554 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5a704-94fc-4857-bd47-f5ba5849721d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:35.576469 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91455d-2495-4075-80c7-3da261508f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:35.577333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91455d-2495-4075-80c7-3da261508f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:35.577617 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91455d-2495-4075-80c7-3da261508f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:35.577892 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91455d-2495-4075-80c7-3da261508f54 admin admin] PATH_INFO: `/v3/projects/b2e6b8e51fc5475ebb09da153aaceaf1` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:35.579125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c91455d-2495-4075-80c7-3da261508f54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2e6b8e51fc5475ebb09da153aaceaf1)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:35.581677 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c91455d-2495-4075-80c7-3da261508f54 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:35.608318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:35.632718 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0c91455d-2495-4075-80c7-3da261508f54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2e6b8e51fc5475ebb09da153aaceaf1', 'request_id': 'req-0c91455d-2495-4075-80c7-3da261508f54'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:35.638761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0c91455d-2495-4075-80c7-3da261508f54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2e6b8e51fc5475ebb09da153aaceaf1', 'request_id': 'req-0c91455d-2495-4075-80c7-3da261508f54'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:35.645178 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0c91455d-2495-4075-80c7-3da261508f54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2e6b8e51fc5475ebb09da153aaceaf1', 'request_id': 'req-0c91455d-2495-4075-80c7-3da261508f54'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:35.657589 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0c91455d-2495-4075-80c7-3da261508f54 admin admin] The token cache is being invalidate because project b2e6b8e51fc5475ebb09da153aaceaf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:35.658086 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0c91455d-2495-4075-80c7-3da261508f54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:35.659492 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0c91455d-2495-4075-80c7-3da261508f54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:35.661079 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1041/2084] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:35 2022] DELETE /identity/v3/projects/b2e6b8e51fc5475ebb09da153aaceaf1 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:35.666365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b3acc-8c67-476a-bb00-2a1aa554bc04 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:35.666699 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b3acc-8c67-476a-bb00-2a1aa554bc04 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:35.666986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b3acc-8c67-476a-bb00-2a1aa554bc04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:35.684372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f93b3acc-8c67-476a-bb00-2a1aa554bc04 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:35.685584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f93b3acc-8c67-476a-bb00-2a1aa554bc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:35.749421 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1044/2085] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:35.757475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c3ab5-0d3f-499f-9ecd-9985f234ea77 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:35.757740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c3ab5-0d3f-499f-9ecd-9985f234ea77 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:35.758028 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c3ab5-0d3f-499f-9ecd-9985f234ea77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:35.778181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ec4c3ab5-0d3f-499f-9ecd-9985f234ea77 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:35.779460 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ec4c3ab5-0d3f-499f-9ecd-9985f234ea77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:35.788319 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1042/2086] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:35.792728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b3acc-8c67-476a-bb00-2a1aa554bc04 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:35.802685 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b3acc-8c67-476a-bb00-2a1aa554bc04 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:35.807228 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d9396-1659-4ae3-8fe6-56b8d9f235de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:35.807895 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d9396-1659-4ae3-8fe6-56b8d9f235de admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:35.808183 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d9396-1659-4ae3-8fe6-56b8d9f235de admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:35.808464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d9396-1659-4ae3-8fe6-56b8d9f235de admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:35.809200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d9396-1659-4ae3-8fe6-56b8d9f235de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:34:35.809512 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d9396-1659-4ae3-8fe6-56b8d9f235de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:35.811166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d9396-1659-4ae3-8fe6-56b8d9f235de admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:35.825043 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1045/2087] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:34:35.829476 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c3ab5-0d3f-499f-9ecd-9985f234ea77 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:35.853235 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c3ab5-0d3f-499f-9ecd-9985f234ea77 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:35.858570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143ebc1-8a34-43f0-804f-0095e1455dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:35.859291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143ebc1-8a34-43f0-804f-0095e1455dc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:35.859582 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143ebc1-8a34-43f0-804f-0095e1455dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:35.859866 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143ebc1-8a34-43f0-804f-0095e1455dc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:35.860686 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143ebc1-8a34-43f0-804f-0095e1455dc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:35.863691 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143ebc1-8a34-43f0-804f-0095e1455dc1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:35.907187 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1043/2088] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:34:35 2022] POST /identity/v3/projects => generated 383 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:35.912701 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d9396-1659-4ae3-8fe6-56b8d9f235de admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:35.932557 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d9396-1659-4ae3-8fe6-56b8d9f235de admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:35.938001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:36.027596 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c1fce-a29d-471f-b809-423b26d27ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:36.028467 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c1fce-a29d-471f-b809-423b26d27ac4 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:36.028851 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c1fce-a29d-471f-b809-423b26d27ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:36.029229 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c1fce-a29d-471f-b809-423b26d27ac4 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:36.030233 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c1fce-a29d-471f-b809-423b26d27ac4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:36.032713 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c1fce-a29d-471f-b809-423b26d27ac4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:36.067948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1046/2089] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:34:35 2022] POST /identity/v3/users => generated 292 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:36.073784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143ebc1-8a34-43f0-804f-0095e1455dc1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:36.090330 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143ebc1-8a34-43f0-804f-0095e1455dc1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:36.095491 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf97235-a70c-47b5-bf02-fa2e05b3205f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:36.096376 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf97235-a70c-47b5-bf02-fa2e05b3205f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:36.096785 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf97235-a70c-47b5-bf02-fa2e05b3205f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:36.097184 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf97235-a70c-47b5-bf02-fa2e05b3205f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:36.097993 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf97235-a70c-47b5-bf02-fa2e05b3205f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:36.100485 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf97235-a70c-47b5-bf02-fa2e05b3205f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:36.117979 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1044/2090] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:36.125250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c1fce-a29d-471f-b809-423b26d27ac4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:36.145545 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c1fce-a29d-471f-b809-423b26d27ac4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:36.153539 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7dfa3c-b69c-4b85-988e-db570f3c982d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:36.154717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7dfa3c-b69c-4b85-988e-db570f3c982d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:36.155244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7dfa3c-b69c-4b85-988e-db570f3c982d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:36.155733 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7dfa3c-b69c-4b85-988e-db570f3c982d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:36.156800 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7dfa3c-b69c-4b85-988e-db570f3c982d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:36.159334 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7dfa3c-b69c-4b85-988e-db570f3c982d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:36.176325 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1047/2091] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:36.176893 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf97235-a70c-47b5-bf02-fa2e05b3205f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:36.178770 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-eaf97235-a70c-47b5-bf02-fa2e05b3205f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:36.180476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7dfa3c-b69c-4b85-988e-db570f3c982d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:36.193917 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf97235-a70c-47b5-bf02-fa2e05b3205f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:36.194736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7dfa3c-b69c-4b85-988e-db570f3c982d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:36.199725 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10817da-c7f3-4980-a7ac-114003cc5b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:36.200596 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10817da-c7f3-4980-a7ac-114003cc5b4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:36.200768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b993312-d468-4be2-bebd-6bc0743f43b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:36.200955 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10817da-c7f3-4980-a7ac-114003cc5b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:36.201139 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10817da-c7f3-4980-a7ac-114003cc5b4b admin admin] PATH_INFO: `/v3/users/0377aca6d6334cb2a764eb583618da98` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:36.201697 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b993312-d468-4be2-bebd-6bc0743f43b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:36.201970 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b993312-d468-4be2-bebd-6bc0743f43b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:36.202320 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10817da-c7f3-4980-a7ac-114003cc5b4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0377aca6d6334cb2a764eb583618da98)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:36.202413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b993312-d468-4be2-bebd-6bc0743f43b6 admin admin] PATH_INFO: `/v3/projects/cb5834cd36864dd9bb27f917f3841cd0/users/afb1a3a42097486ea1011b12ece8d435/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:36.204398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10817da-c7f3-4980-a7ac-114003cc5b4b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:36.224945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b993312-d468-4be2-bebd-6bc0743f43b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb5834cd36864dd9bb27f917f3841cd0, user_id=afb1a3a42097486ea1011b12ece8d435, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:36.229105 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b993312-d468-4be2-bebd-6bc0743f43b6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:36.249768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1048/2092] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:36 2022] PUT /identity/v3/projects/cb5834cd36864dd9bb27f917f3841cd0/users/afb1a3a42097486ea1011b12ece8d435/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:34:36.263320 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c10817da-c7f3-4980-a7ac-114003cc5b4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0377aca6d6334cb2a764eb583618da98'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:36.263581 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456ec92-aa37-402d-a2d5-832229cdbdfc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:36.263975 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456ec92-aa37-402d-a2d5-832229cdbdfc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:36.264328 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456ec92-aa37-402d-a2d5-832229cdbdfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:36.272320 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c10817da-c7f3-4980-a7ac-114003cc5b4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0377aca6d6334cb2a764eb583618da98'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:36.292773 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c10817da-c7f3-4980-a7ac-114003cc5b4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0377aca6d6334cb2a764eb583618da98'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:36.293978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c10817da-c7f3-4980-a7ac-114003cc5b4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0377aca6d6334cb2a764eb583618da98'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:36.294720 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c10817da-c7f3-4980-a7ac-114003cc5b4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0377aca6d6334cb2a764eb583618da98'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:36.295541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e456ec92-aa37-402d-a2d5-832229cdbdfc None None] MFA Rules not processed for user `afb1a3a42097486ea1011b12ece8d435`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:36.296771 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e456ec92-aa37-402d-a2d5-832229cdbdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:36.313412 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1045/2093] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:34:36 2022] DELETE /identity/v3/users/0377aca6d6334cb2a764eb583618da98 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:34:36.320623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5ce04-f8e7-4b83-97ff-362050985d2f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:36.320815 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5ce04-f8e7-4b83-97ff-362050985d2f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:36.321090 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5ce04-f8e7-4b83-97ff-362050985d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:36.350104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e8f5ce04-f8e7-4b83-97ff-362050985d2f None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:36.350104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e8f5ce04-f8e7-4b83-97ff-362050985d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:36.390703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1049/2094] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:34:36.396609 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e456ec92-aa37-402d-a2d5-832229cdbdfc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:36.398622 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e456ec92-aa37-402d-a2d5-832229cdbdfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:36.401960 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=17,keystone:SELECT=80,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:36.430588 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1046/2095] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:36.441172 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f5ce04-f8e7-4b83-97ff-362050985d2f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:36.442720 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e8f5ce04-f8e7-4b83-97ff-362050985d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:36.490788 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:36.530364 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e456ec92-aa37-402d-a2d5-832229cdbdfc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:36.530730 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f5ce04-f8e7-4b83-97ff-362050985d2f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:36.537418 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53d3fd-6640-4c13-968f-ed1641d9c8b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:36.539522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53d3fd-6640-4c13-968f-ed1641d9c8b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:36.539815 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53d3fd-6640-4c13-968f-ed1641d9c8b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:36.540099 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53d3fd-6640-4c13-968f-ed1641d9c8b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:36.541568 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a42cf-5b17-4455-9aae-16169e852883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:36.542239 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a42cf-5b17-4455-9aae-16169e852883 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:36.542695 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a42cf-5b17-4455-9aae-16169e852883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:36.542893 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a42cf-5b17-4455-9aae-16169e852883 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:36.543532 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49a42cf-5b17-4455-9aae-16169e852883 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:36.545446 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49a42cf-5b17-4455-9aae-16169e852883 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:36.562337 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f53d3fd-6640-4c13-968f-ed1641d9c8b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:36.564249 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f53d3fd-6640-4c13-968f-ed1641d9c8b4 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:36.583973 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1050/2096] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:34:36 2022] POST /identity/v3/projects => generated 383 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:34:36.585025 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c2a4f-de50-44f8-95ec-00e699347d5d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:36.585421 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c2a4f-de50-44f8-95ec-00e699347d5d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:36.586118 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c2a4f-de50-44f8-95ec-00e699347d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:36.586643 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1047/2097] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:36.588179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53d3fd-6640-4c13-968f-ed1641d9c8b4 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:36.600457 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53d3fd-6640-4c13-968f-ed1641d9c8b4 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:36.612432 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-241c2a4f-de50-44f8-95ec-00e699347d5d None None] MFA Rules not processed for user `bba6a17fc19645c29fce6fa655095a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:36.613692 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-241c2a4f-de50-44f8-95ec-00e699347d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:36.687442 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1051/2098] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:36.693706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6eac5-0c54-4598-a500-5e313561c2c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:36.694864 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6eac5-0c54-4598-a500-5e313561c2c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:36.695259 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6eac5-0c54-4598-a500-5e313561c2c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:36.696226 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6eac5-0c54-4598-a500-5e313561c2c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:36.696524 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6eac5-0c54-4598-a500-5e313561c2c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:36.699677 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6eac5-0c54-4598-a500-5e313561c2c8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:36.732387 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1048/2099] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:34:36 2022] POST /identity/v3/users => generated 292 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:36.737140 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c2a4f-de50-44f8-95ec-00e699347d5d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:36.751915 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c2a4f-de50-44f8-95ec-00e699347d5d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:36.757488 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c4cff-3cf8-41e2-a6e8-7e42b854f070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:36.758271 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c4cff-3cf8-41e2-a6e8-7e42b854f070 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:36.758679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c4cff-3cf8-41e2-a6e8-7e42b854f070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:36.759057 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c4cff-3cf8-41e2-a6e8-7e42b854f070 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:36.759895 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c4cff-3cf8-41e2-a6e8-7e42b854f070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:36.761898 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c4cff-3cf8-41e2-a6e8-7e42b854f070 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:36.777416 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1052/2100] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:36.778721 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6eac5-0c54-4598-a500-5e313561c2c8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:36.791204 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6eac5-0c54-4598-a500-5e313561c2c8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:36.797096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef5b8f-6220-41b7-834f-dc404cbb7f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:36.797954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef5b8f-6220-41b7-834f-dc404cbb7f41 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:36.798345 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef5b8f-6220-41b7-834f-dc404cbb7f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:36.798726 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef5b8f-6220-41b7-834f-dc404cbb7f41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:36.799902 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef5b8f-6220-41b7-834f-dc404cbb7f41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:36.801709 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef5b8f-6220-41b7-834f-dc404cbb7f41 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:36.818475 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1049/2101] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:36.821967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c4cff-3cf8-41e2-a6e8-7e42b854f070 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:36.836067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c4cff-3cf8-41e2-a6e8-7e42b854f070 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:36.844204 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d9f58-32a6-4b73-b562-eb7ebbef2db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:36.845265 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d9f58-32a6-4b73-b562-eb7ebbef2db1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:36.845602 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d9f58-32a6-4b73-b562-eb7ebbef2db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:36.847080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d9f58-32a6-4b73-b562-eb7ebbef2db1 admin admin] PATH_INFO: `/v3/projects/159edb8bb6b44df5a4dc88e0bdeab421/users/7186b29af5c7494a891b303887b27742/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:36.865961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d9f58-32a6-4b73-b562-eb7ebbef2db1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=159edb8bb6b44df5a4dc88e0bdeab421, user_id=7186b29af5c7494a891b303887b27742, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:36.868921 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d9f58-32a6-4b73-b562-eb7ebbef2db1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:36.904369 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1053/2102] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:36 2022] PUT /identity/v3/projects/159edb8bb6b44df5a4dc88e0bdeab421/users/7186b29af5c7494a891b303887b27742/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:36.916281 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b503a-90b2-43d2-b061-7c969ae8749e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:36.916638 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b503a-90b2-43d2-b061-7c969ae8749e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:36.916985 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8b503a-90b2-43d2-b061-7c969ae8749e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:36.954789 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ff8b503a-90b2-43d2-b061-7c969ae8749e None None] MFA Rules not processed for user `7186b29af5c7494a891b303887b27742`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:36.957282 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ff8b503a-90b2-43d2-b061-7c969ae8749e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:37.044910 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:37.064454 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1050/2103] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:37.070396 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d9f58-32a6-4b73-b562-eb7ebbef2db1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:37.151005 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d9f58-32a6-4b73-b562-eb7ebbef2db1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:37.163196 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c14d1-8fba-4a0c-8f5f-503007834051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:37.164423 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c14d1-8fba-4a0c-8f5f-503007834051 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:37.164921 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c14d1-8fba-4a0c-8f5f-503007834051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:37.165917 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c14d1-8fba-4a0c-8f5f-503007834051 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:37.167399 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c14d1-8fba-4a0c-8f5f-503007834051 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:37.170695 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c14d1-8fba-4a0c-8f5f-503007834051 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:37.199469 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b503a-90b2-43d2-b061-7c969ae8749e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:37.201086 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ff8b503a-90b2-43d2-b061-7c969ae8749e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:37.218326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1054/2104] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:34:37 2022] POST /identity/v3/projects => generated 383 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:34:37.223386 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c14d1-8fba-4a0c-8f5f-503007834051 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:37.227167 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8b503a-90b2-43d2-b061-7c969ae8749e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:37.275429 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:37.315634 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c14d1-8fba-4a0c-8f5f-503007834051 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:37.321039 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05744e-453d-495a-8a4e-7ef2165ed076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:37.322458 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05744e-453d-495a-8a4e-7ef2165ed076 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:37.322458 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05744e-453d-495a-8a4e-7ef2165ed076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:37.322458 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05744e-453d-495a-8a4e-7ef2165ed076 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:37.322734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c729a6-339c-4e65-ac30-d56d2ff64795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:37.323084 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b05744e-453d-495a-8a4e-7ef2165ed076 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:37.323606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c729a6-339c-4e65-ac30-d56d2ff64795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:37.325327 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b05744e-453d-495a-8a4e-7ef2165ed076 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:37.325490 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c729a6-339c-4e65-ac30-d56d2ff64795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:37.325802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c729a6-339c-4e65-ac30-d56d2ff64795 admin admin] PATH_INFO: `/v3/projects/7c45bf36ed544f0a9f5ee87df819c7d0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:37.327091 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c729a6-339c-4e65-ac30-d56d2ff64795 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c45bf36ed544f0a9f5ee87df819c7d0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:37.330022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c729a6-339c-4e65-ac30-d56d2ff64795 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:37.355099 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1055/2105] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:34:37 2022] POST /identity/v3/users => generated 298 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:34:37.359496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05744e-453d-495a-8a4e-7ef2165ed076 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:37.370732 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05744e-453d-495a-8a4e-7ef2165ed076 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:37.377734 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01580c33-d0cf-41c4-b375-ec38493b1595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:37.378745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01580c33-d0cf-41c4-b375-ec38493b1595 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:37.379127 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01580c33-d0cf-41c4-b375-ec38493b1595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:37.379414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01580c33-d0cf-41c4-b375-ec38493b1595 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:37.380229 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01580c33-d0cf-41c4-b375-ec38493b1595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:37.384639 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01580c33-d0cf-41c4-b375-ec38493b1595 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:37.398805 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1056/2106] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:37.403142 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f0c729a6-339c-4e65-ac30-d56d2ff64795 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c45bf36ed544f0a9f5ee87df819c7d0', 'request_id': 'req-f0c729a6-339c-4e65-ac30-d56d2ff64795'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:37.405744 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01580c33-d0cf-41c4-b375-ec38493b1595 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:37.412921 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f0c729a6-339c-4e65-ac30-d56d2ff64795 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c45bf36ed544f0a9f5ee87df819c7d0', 'request_id': 'req-f0c729a6-339c-4e65-ac30-d56d2ff64795'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:37.420787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01580c33-d0cf-41c4-b375-ec38493b1595 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:37.422484 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f0c729a6-339c-4e65-ac30-d56d2ff64795 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c45bf36ed544f0a9f5ee87df819c7d0', 'request_id': 'req-f0c729a6-339c-4e65-ac30-d56d2ff64795'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:37.444735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f0c729a6-339c-4e65-ac30-d56d2ff64795 admin admin] The token cache is being invalidate because project 7c45bf36ed544f0a9f5ee87df819c7d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:37.445104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f0c729a6-339c-4e65-ac30-d56d2ff64795 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:37.445920 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f0c729a6-339c-4e65-ac30-d56d2ff64795 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:37.454561 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1051/2107] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:37 2022] DELETE /identity/v3/projects/7c45bf36ed544f0a9f5ee87df819c7d0 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:37.456095 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c729a6-339c-4e65-ac30-d56d2ff64795 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:37.457803 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f0c729a6-339c-4e65-ac30-d56d2ff64795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:37.515535 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a71b3-8ffc-4aaa-9b29-30c5af4f73c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:37.517040 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a71b3-8ffc-4aaa-9b29-30c5af4f73c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:37.517473 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a71b3-8ffc-4aaa-9b29-30c5af4f73c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:37.520876 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a71b3-8ffc-4aaa-9b29-30c5af4f73c6 admin admin] PATH_INFO: `/v3/projects/21353f0266d84c96af54e3778c63c77d/users/3104e8fd088040878d151b3b5dc42682/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:37.544580 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a71b3-8ffc-4aaa-9b29-30c5af4f73c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21353f0266d84c96af54e3778c63c77d, user_id=3104e8fd088040878d151b3b5dc42682, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:37.549788 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a71b3-8ffc-4aaa-9b29-30c5af4f73c6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:37.562007 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c729a6-339c-4e65-ac30-d56d2ff64795 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:37.568244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1057/2108] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:37 2022] PUT /identity/v3/projects/21353f0266d84c96af54e3778c63c77d/users/3104e8fd088040878d151b3b5dc42682/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:37.572538 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a71b3-8ffc-4aaa-9b29-30c5af4f73c6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:37.575213 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-364a71b3-8ffc-4aaa-9b29-30c5af4f73c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:37.601141 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:37.641332 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16ce57-4229-4be0-8d6f-eee56bacf6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:37.642377 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16ce57-4229-4be0-8d6f-eee56bacf6a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:37.642956 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16ce57-4229-4be0-8d6f-eee56bacf6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:37.644157 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b16ce57-4229-4be0-8d6f-eee56bacf6a4 admin admin] PATH_INFO: `/v3/users/867fedeb546449fdaa9292e39befaa49` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:37.647675 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b16ce57-4229-4be0-8d6f-eee56bacf6a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=867fedeb546449fdaa9292e39befaa49)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:37.647675 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b16ce57-4229-4be0-8d6f-eee56bacf6a4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:37.667438 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a71b3-8ffc-4aaa-9b29-30c5af4f73c6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:37.672532 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890b002-fd9e-48de-8fd8-12c62f8a889c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:37.673353 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890b002-fd9e-48de-8fd8-12c62f8a889c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:37.673744 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890b002-fd9e-48de-8fd8-12c62f8a889c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:37.674667 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5890b002-fd9e-48de-8fd8-12c62f8a889c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:37.676471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890b002-fd9e-48de-8fd8-12c62f8a889c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:37.678442 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5890b002-fd9e-48de-8fd8-12c62f8a889c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:37.694405 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1058/2109] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:37 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:34:37.708300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890b002-fd9e-48de-8fd8-12c62f8a889c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:37.714870 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1b16ce57-4229-4be0-8d6f-eee56bacf6a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '867fedeb546449fdaa9292e39befaa49'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:37.722705 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5890b002-fd9e-48de-8fd8-12c62f8a889c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:37.725611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1b16ce57-4229-4be0-8d6f-eee56bacf6a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '867fedeb546449fdaa9292e39befaa49'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:37.728670 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d70cd-1131-488e-9580-d690297a0cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:37.730037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d70cd-1131-488e-9580-d690297a0cbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:37.730336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d70cd-1131-488e-9580-d690297a0cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:37.730814 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d70cd-1131-488e-9580-d690297a0cbf admin admin] PATH_INFO: `/v3/projects/21353f0266d84c96af54e3778c63c77d/users/3104e8fd088040878d151b3b5dc42682/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:37.733021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d70cd-1131-488e-9580-d690297a0cbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=21353f0266d84c96af54e3778c63c77d, user_id=3104e8fd088040878d151b3b5dc42682, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:37.736266 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d70cd-1131-488e-9580-d690297a0cbf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:37.745791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1b16ce57-4229-4be0-8d6f-eee56bacf6a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '867fedeb546449fdaa9292e39befaa49'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:37.746638 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1b16ce57-4229-4be0-8d6f-eee56bacf6a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '867fedeb546449fdaa9292e39befaa49'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:37.747491 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1b16ce57-4229-4be0-8d6f-eee56bacf6a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '867fedeb546449fdaa9292e39befaa49'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:37.755094 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1059/2110] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:37 2022] PUT /identity/v3/projects/21353f0266d84c96af54e3778c63c77d/users/3104e8fd088040878d151b3b5dc42682/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:37.762480 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1052/2111] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:34:37 2022] DELETE /identity/v3/users/867fedeb546449fdaa9292e39befaa49 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:37.769777 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a70882-ef4f-460c-9daa-305bb6c7acbf None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:37.770604 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a70882-ef4f-460c-9daa-305bb6c7acbf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:37.771265 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a70882-ef4f-460c-9daa-305bb6c7acbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:37.805088 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d4a70882-ef4f-460c-9daa-305bb6c7acbf None None] MFA Rules not processed for user `3104e8fd088040878d151b3b5dc42682`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:37.806540 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d4a70882-ef4f-460c-9daa-305bb6c7acbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:37.810845 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:37.904236 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1060/2112] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:37.920984 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16ce57-4229-4be0-8d6f-eee56bacf6a4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:37.927041 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1b16ce57-4229-4be0-8d6f-eee56bacf6a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:38.005703 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b16ce57-4229-4be0-8d6f-eee56bacf6a4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:38.009490 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d701f073-0295-4c22-baaa-3ea83626f0f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:38.010302 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701f073-0295-4c22-baaa-3ea83626f0f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:38.010690 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701f073-0295-4c22-baaa-3ea83626f0f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:38.010974 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d701f073-0295-4c22-baaa-3ea83626f0f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:38.012779 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d701f073-0295-4c22-baaa-3ea83626f0f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:38.079662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d701f073-0295-4c22-baaa-3ea83626f0f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:38.081325 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d701f073-0295-4c22-baaa-3ea83626f0f3 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:38.092553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1053/2113] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:38.194707 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a70882-ef4f-460c-9daa-305bb6c7acbf None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:38.196294 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d4a70882-ef4f-460c-9daa-305bb6c7acbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:38.307018 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a70882-ef4f-460c-9daa-305bb6c7acbf None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:38.313031 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683b484-fa88-4b88-a43a-5852a01974ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:38.314168 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683b484-fa88-4b88-a43a-5852a01974ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:38.314658 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683b484-fa88-4b88-a43a-5852a01974ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:38.315770 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683b484-fa88-4b88-a43a-5852a01974ff admin admin] PATH_INFO: `/v3/projects/053e29d4ac0f47e8956d3f5d7a1b4598` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:38.317504 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3683b484-fa88-4b88-a43a-5852a01974ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=053e29d4ac0f47e8956d3f5d7a1b4598)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:38.321504 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3683b484-fa88-4b88-a43a-5852a01974ff admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:38.388378 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3683b484-fa88-4b88-a43a-5852a01974ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '053e29d4ac0f47e8956d3f5d7a1b4598', 'request_id': 'req-3683b484-fa88-4b88-a43a-5852a01974ff'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:38.397915 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3683b484-fa88-4b88-a43a-5852a01974ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '053e29d4ac0f47e8956d3f5d7a1b4598', 'request_id': 'req-3683b484-fa88-4b88-a43a-5852a01974ff'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:38.406461 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3683b484-fa88-4b88-a43a-5852a01974ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '053e29d4ac0f47e8956d3f5d7a1b4598', 'request_id': 'req-3683b484-fa88-4b88-a43a-5852a01974ff'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:38.421674 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3683b484-fa88-4b88-a43a-5852a01974ff admin admin] The token cache is being invalidate because project 053e29d4ac0f47e8956d3f5d7a1b4598 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:38.422201 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3683b484-fa88-4b88-a43a-5852a01974ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:38.423072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3683b484-fa88-4b88-a43a-5852a01974ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:38.430411 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1061/2114] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:38 2022] DELETE /identity/v3/projects/053e29d4ac0f47e8956d3f5d7a1b4598 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:38.431465 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2ba96-531b-4a61-bd2a-bc05a6fe9c23 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:38.432127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2ba96-531b-4a61-bd2a-bc05a6fe9c23 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:38.432127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2ba96-531b-4a61-bd2a-bc05a6fe9c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:38.442553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:38.457050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bdb2ba96-531b-4a61-bd2a-bc05a6fe9c23 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:38.458901 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bdb2ba96-531b-4a61-bd2a-bc05a6fe9c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:38.539627 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1054/2115] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:38.547494 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000dae9-f19a-4759-9938-c9b1aa751459 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:38.547798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000dae9-f19a-4759-9938-c9b1aa751459 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:38.548087 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000dae9-f19a-4759-9938-c9b1aa751459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:38.567644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b000dae9-f19a-4759-9938-c9b1aa751459 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:38.568858 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b000dae9-f19a-4759-9938-c9b1aa751459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:38.577908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1062/2116] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:38.582968 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2ba96-531b-4a61-bd2a-bc05a6fe9c23 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:38.595628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2ba96-531b-4a61-bd2a-bc05a6fe9c23 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:38.601407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4bad1a-c501-4282-af4f-5984fd873146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:38.602251 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bad1a-c501-4282-af4f-5984fd873146 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:38.602788 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bad1a-c501-4282-af4f-5984fd873146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:38.603166 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4bad1a-c501-4282-af4f-5984fd873146 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:38.604001 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4bad1a-c501-4282-af4f-5984fd873146 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:34:38.604505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4bad1a-c501-4282-af4f-5984fd873146 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:38.606279 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4bad1a-c501-4282-af4f-5984fd873146 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:38.624147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1055/2117] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:34:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:34:38.625818 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b000dae9-f19a-4759-9938-c9b1aa751459 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:38.637158 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b000dae9-f19a-4759-9938-c9b1aa751459 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:38.642000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375af4f-b3ae-488c-ad9d-2b0cd469abb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:38.642826 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375af4f-b3ae-488c-ad9d-2b0cd469abb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:38.643190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375af4f-b3ae-488c-ad9d-2b0cd469abb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:38.643648 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375af4f-b3ae-488c-ad9d-2b0cd469abb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:38.644436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9375af4f-b3ae-488c-ad9d-2b0cd469abb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:38.646705 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9375af4f-b3ae-488c-ad9d-2b0cd469abb7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:38.683746 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1063/2118] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:34:38 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:38.688125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4bad1a-c501-4282-af4f-5984fd873146 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:38.699737 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4bad1a-c501-4282-af4f-5984fd873146 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:38.761550 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b263ddae-3891-4448-9ac1-adb76a651045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:38.762239 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263ddae-3891-4448-9ac1-adb76a651045 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:38.762562 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263ddae-3891-4448-9ac1-adb76a651045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:38.763089 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263ddae-3891-4448-9ac1-adb76a651045 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:38.763810 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b263ddae-3891-4448-9ac1-adb76a651045 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:38.765793 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b263ddae-3891-4448-9ac1-adb76a651045 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:38.797026 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375af4f-b3ae-488c-ad9d-2b0cd469abb7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:38.800141 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1056/2119] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:34:38 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:38.803826 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:38.812198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375af4f-b3ae-488c-ad9d-2b0cd469abb7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:38.820753 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb86a8e-bd3e-493e-a37b-9bf45a11253e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:38.820753 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb86a8e-bd3e-493e-a37b-9bf45a11253e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:38.820753 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb86a8e-bd3e-493e-a37b-9bf45a11253e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:38.820753 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb86a8e-bd3e-493e-a37b-9bf45a11253e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:38.821418 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb86a8e-bd3e-493e-a37b-9bf45a11253e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:38.823088 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb86a8e-bd3e-493e-a37b-9bf45a11253e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:38.834593 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1064/2120] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:38.838932 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b263ddae-3891-4448-9ac1-adb76a651045 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:38.849891 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b263ddae-3891-4448-9ac1-adb76a651045 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:38.854567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48722ac-10ce-48c5-9c6d-e9f2106ad8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:38.855227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48722ac-10ce-48c5-9c6d-e9f2106ad8b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:38.855516 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48722ac-10ce-48c5-9c6d-e9f2106ad8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:38.855798 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48722ac-10ce-48c5-9c6d-e9f2106ad8b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:38.856790 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48722ac-10ce-48c5-9c6d-e9f2106ad8b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:38.858412 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48722ac-10ce-48c5-9c6d-e9f2106ad8b9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:38.869219 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1057/2121] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:38.876820 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb86a8e-bd3e-493e-a37b-9bf45a11253e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:38.888099 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb86a8e-bd3e-493e-a37b-9bf45a11253e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:38.892576 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de6f3a-616f-4ea9-aac0-ade69d3fcaa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:38.893681 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de6f3a-616f-4ea9-aac0-ade69d3fcaa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:38.893969 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de6f3a-616f-4ea9-aac0-ade69d3fcaa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:38.894247 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de6f3a-616f-4ea9-aac0-ade69d3fcaa3 admin admin] PATH_INFO: `/v3/projects/698adaaf9d124670be6ccdcb3cbd7ece/users/fa9383b7671347e98bdc481c2d0c6486/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:38.909633 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de6f3a-616f-4ea9-aac0-ade69d3fcaa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=698adaaf9d124670be6ccdcb3cbd7ece, user_id=fa9383b7671347e98bdc481c2d0c6486, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:38.912872 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de6f3a-616f-4ea9-aac0-ade69d3fcaa3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:38.926952 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1065/2122] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:38 2022] PUT /identity/v3/projects/698adaaf9d124670be6ccdcb3cbd7ece/users/fa9383b7671347e98bdc481c2d0c6486/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:38.932524 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e9839-e14c-43eb-9238-e94ced6d10fd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:38.932524 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e9839-e14c-43eb-9238-e94ced6d10fd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:38.933048 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e9839-e14c-43eb-9238-e94ced6d10fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:38.954116 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-aa6e9839-e14c-43eb-9238-e94ced6d10fd None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:38.955443 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-aa6e9839-e14c-43eb-9238-e94ced6d10fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:39.019972 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1058/2123] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:34:39.029868 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de6f3a-616f-4ea9-aac0-ade69d3fcaa3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:39.031623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-35de6f3a-616f-4ea9-aac0-ade69d3fcaa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:39.091184 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de6f3a-616f-4ea9-aac0-ade69d3fcaa3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:39.094885 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4591c-0b26-41ca-82bd-78e2c2dbb621 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:39.095545 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4591c-0b26-41ca-82bd-78e2c2dbb621 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:39.096381 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4591c-0b26-41ca-82bd-78e2c2dbb621 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:39.097198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4591c-0b26-41ca-82bd-78e2c2dbb621 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:39.107131 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4591c-0b26-41ca-82bd-78e2c2dbb621 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:39.108955 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4591c-0b26-41ca-82bd-78e2c2dbb621 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:39.120113 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1066/2124] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:39.844267 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e9839-e14c-43eb-9238-e94ced6d10fd None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:39.846238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-aa6e9839-e14c-43eb-9238-e94ced6d10fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:39.860874 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:39.916310 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e9839-e14c-43eb-9238-e94ced6d10fd None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:39.920292 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e641d5e-a5b9-426e-9670-4640b7d3ca5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:39.920979 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e641d5e-a5b9-426e-9670-4640b7d3ca5f service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:39.921270 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e641d5e-a5b9-426e-9670-4640b7d3ca5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:39.921556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e641d5e-a5b9-426e-9670-4640b7d3ca5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:39.923387 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0e641d5e-a5b9-426e-9670-4640b7d3ca5f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:39.990019 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e641d5e-a5b9-426e-9670-4640b7d3ca5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:39.991528 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e641d5e-a5b9-426e-9670-4640b7d3ca5f service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:40.003639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1059/2125] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:40.669635 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ce7d1-ddc8-4e8f-822c-ba70d4cbe0b0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:40.670846 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ce7d1-ddc8-4e8f-822c-ba70d4cbe0b0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:40.671730 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ce7d1-ddc8-4e8f-822c-ba70d4cbe0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:40.707732 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9c1ce7d1-ddc8-4e8f-822c-ba70d4cbe0b0 None None] MFA Rules not processed for user `fa9383b7671347e98bdc481c2d0c6486`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:40.709090 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9c1ce7d1-ddc8-4e8f-822c-ba70d4cbe0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:40.800614 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1067/2126] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:40.802251 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e641d5e-a5b9-426e-9670-4640b7d3ca5f service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:40.820845 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e641d5e-a5b9-426e-9670-4640b7d3ca5f service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:40.822748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ce7d1-ddc8-4e8f-822c-ba70d4cbe0b0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:40.827559 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b1671-2b6a-4840-a02b-5f4e87153e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:40.828381 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b1671-2b6a-4840-a02b-5f4e87153e6a admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:40.828768 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b1671-2b6a-4840-a02b-5f4e87153e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:40.829169 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b1671-2b6a-4840-a02b-5f4e87153e6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:40.829947 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b1671-2b6a-4840-a02b-5f4e87153e6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:40.833173 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b1671-2b6a-4840-a02b-5f4e87153e6a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:40.835587 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ce7d1-ddc8-4e8f-822c-ba70d4cbe0b0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:40.840282 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4ad96-9a2b-46b7-b0de-821a66279747 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:40.841596 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4ad96-9a2b-46b7-b0de-821a66279747 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:40.842061 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4ad96-9a2b-46b7-b0de-821a66279747 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:40.842351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4ad96-9a2b-46b7-b0de-821a66279747 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:40.845120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6cd4ad96-9a2b-46b7-b0de-821a66279747 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:40.866129 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1060/2127] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:34:40 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:40.869898 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b1671-2b6a-4840-a02b-5f4e87153e6a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:40.965008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:40.969599 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4ad96-9a2b-46b7-b0de-821a66279747 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:40.972037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4ad96-9a2b-46b7-b0de-821a66279747 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:40.988050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b1671-2b6a-4840-a02b-5f4e87153e6a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:40.990000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1068/2128] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:41.027680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e6c7a-7fde-481a-948c-0ce1391e73cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:41.028391 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e6c7a-7fde-481a-948c-0ce1391e73cc admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:41.028673 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e6c7a-7fde-481a-948c-0ce1391e73cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:41.028959 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e6c7a-7fde-481a-948c-0ce1391e73cc admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:41.030337 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e6c7a-7fde-481a-948c-0ce1391e73cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:41.032331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2e6c7a-7fde-481a-948c-0ce1391e73cc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:41.085033 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1061/2129] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:34:40 2022] POST /identity/v3/users => generated 292 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:41.089214 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4ad96-9a2b-46b7-b0de-821a66279747 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:41.104821 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4ad96-9a2b-46b7-b0de-821a66279747 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:41.110333 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b715879f-af0c-45f9-a05c-a9172b83142b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:41.111105 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715879f-af0c-45f9-a05c-a9172b83142b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:41.111421 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715879f-af0c-45f9-a05c-a9172b83142b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:41.112307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b715879f-af0c-45f9-a05c-a9172b83142b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:41.113146 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b715879f-af0c-45f9-a05c-a9172b83142b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:41.114907 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b715879f-af0c-45f9-a05c-a9172b83142b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:41.131834 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1069/2130] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:34:41.135904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e6c7a-7fde-481a-948c-0ce1391e73cc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:41.152598 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2e6c7a-7fde-481a-948c-0ce1391e73cc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:41.165571 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf31c35-7471-481d-ad39-bb77445790fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:41.166567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf31c35-7471-481d-ad39-bb77445790fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:41.167338 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf31c35-7471-481d-ad39-bb77445790fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:41.167627 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf31c35-7471-481d-ad39-bb77445790fe admin admin] PATH_INFO: `/v3/projects/6026d9a909dc492cadfc3d899085940e/users/027c1ae7597640929980c680ecc071a4/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:41.204185 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf31c35-7471-481d-ad39-bb77445790fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=6026d9a909dc492cadfc3d899085940e, user_id=027c1ae7597640929980c680ecc071a4, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:41.207282 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf31c35-7471-481d-ad39-bb77445790fe admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:41.225845 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1062/2131] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:41 2022] PUT /identity/v3/projects/6026d9a909dc492cadfc3d899085940e/users/027c1ae7597640929980c680ecc071a4/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:41.230596 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b715879f-af0c-45f9-a05c-a9172b83142b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:41.244024 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b715879f-af0c-45f9-a05c-a9172b83142b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:41.312263 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d45944-c814-4466-b0b0-62fa7939c230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:41.313015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d45944-c814-4466-b0b0-62fa7939c230 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:41.313286 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d45944-c814-4466-b0b0-62fa7939c230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:41.313829 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d45944-c814-4466-b0b0-62fa7939c230 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:41.314359 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d45944-c814-4466-b0b0-62fa7939c230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:41.317262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d45944-c814-4466-b0b0-62fa7939c230 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:41.329300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1070/2132] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:41.334131 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf31c35-7471-481d-ad39-bb77445790fe admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:41.353976 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf31c35-7471-481d-ad39-bb77445790fe admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:41.363217 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f887b9ee-fb9b-4b74-9971-47c5c55ec2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:41.365063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f887b9ee-fb9b-4b74-9971-47c5c55ec2bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:41.366148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f887b9ee-fb9b-4b74-9971-47c5c55ec2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:41.366586 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f887b9ee-fb9b-4b74-9971-47c5c55ec2bf admin admin] PATH_INFO: `/v3/projects/6026d9a909dc492cadfc3d899085940e/users/027c1ae7597640929980c680ecc071a4/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:41.369900 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f887b9ee-fb9b-4b74-9971-47c5c55ec2bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=6026d9a909dc492cadfc3d899085940e, user_id=027c1ae7597640929980c680ecc071a4, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:41.374061 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f887b9ee-fb9b-4b74-9971-47c5c55ec2bf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:41.378861 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:41.390012 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1063/2133] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:41 2022] PUT /identity/v3/projects/6026d9a909dc492cadfc3d899085940e/users/027c1ae7597640929980c680ecc071a4/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:42.567122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01cebb-4917-4f63-b38e-8e57ecaaeb97 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:42.567620 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01cebb-4917-4f63-b38e-8e57ecaaeb97 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:42.567797 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01cebb-4917-4f63-b38e-8e57ecaaeb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:42.605517 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5c01cebb-4917-4f63-b38e-8e57ecaaeb97 None None] MFA Rules not processed for user `027c1ae7597640929980c680ecc071a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:42.607867 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5c01cebb-4917-4f63-b38e-8e57ecaaeb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:42.701710 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1071/2134] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:42 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:42.709537 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f887b9ee-fb9b-4b74-9971-47c5c55ec2bf admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:42.787135 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f887b9ee-fb9b-4b74-9971-47c5c55ec2bf admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:42.791428 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a46b5-6306-4205-9640-ed7639ca4bd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:42.792091 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a46b5-6306-4205-9640-ed7639ca4bd1 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:42.792410 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a46b5-6306-4205-9640-ed7639ca4bd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:42.792644 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930a46b5-6306-4205-9640-ed7639ca4bd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:42.860959 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a46b5-6306-4205-9640-ed7639ca4bd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:42.863572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930a46b5-6306-4205-9640-ed7639ca4bd1 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:42.883952 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1064/2135] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:42 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:44.292128 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c01cebb-4917-4f63-b38e-8e57ecaaeb97 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:44.302958 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c01cebb-4917-4f63-b38e-8e57ecaaeb97 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:44.309127 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4574f4f-3a06-4c27-8109-94108fc55be5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:44.309795 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4574f4f-3a06-4c27-8109-94108fc55be5 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:44.310254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4574f4f-3a06-4c27-8109-94108fc55be5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:44.310758 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4574f4f-3a06-4c27-8109-94108fc55be5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:44.322224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4574f4f-3a06-4c27-8109-94108fc55be5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:44.323731 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4574f4f-3a06-4c27-8109-94108fc55be5 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:44.335443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1072/2136] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:44 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:45.037382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a46b5-6306-4205-9640-ed7639ca4bd1 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:45.038873 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-930a46b5-6306-4205-9640-ed7639ca4bd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:45.118559 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930a46b5-6306-4205-9640-ed7639ca4bd1 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:45.124978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d296f-654e-401c-8ad2-3ec4f96be424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:45.126214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d296f-654e-401c-8ad2-3ec4f96be424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:45.126693 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d296f-654e-401c-8ad2-3ec4f96be424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:45.127209 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d296f-654e-401c-8ad2-3ec4f96be424 admin admin] PATH_INFO: `/v3/users/afb1a3a42097486ea1011b12ece8d435` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:45.129191 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1d296f-654e-401c-8ad2-3ec4f96be424 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afb1a3a42097486ea1011b12ece8d435)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:45.132459 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1d296f-654e-401c-8ad2-3ec4f96be424 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:45.202163 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8f1d296f-654e-401c-8ad2-3ec4f96be424 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afb1a3a42097486ea1011b12ece8d435'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:45.207118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:45.212791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8f1d296f-654e-401c-8ad2-3ec4f96be424 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afb1a3a42097486ea1011b12ece8d435'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:45.230072 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8f1d296f-654e-401c-8ad2-3ec4f96be424 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afb1a3a42097486ea1011b12ece8d435'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:45.231194 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8f1d296f-654e-401c-8ad2-3ec4f96be424 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afb1a3a42097486ea1011b12ece8d435'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:45.231952 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8f1d296f-654e-401c-8ad2-3ec4f96be424 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afb1a3a42097486ea1011b12ece8d435'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:45.246158 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1065/2137] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:34:45 2022] DELETE /identity/v3/users/afb1a3a42097486ea1011b12ece8d435 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:45.250922 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0ea6e-839f-4b79-b78c-8b5cae4508b2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:45.251216 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0ea6e-839f-4b79-b78c-8b5cae4508b2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:45.251495 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0ea6e-839f-4b79-b78c-8b5cae4508b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:45.270479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-83a0ea6e-839f-4b79-b78c-8b5cae4508b2 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:45.271977 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-83a0ea6e-839f-4b79-b78c-8b5cae4508b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:45.325277 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:45.342557 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1073/2138] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:45.353392 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d296f-654e-401c-8ad2-3ec4f96be424 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:45.354981 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8f1d296f-654e-401c-8ad2-3ec4f96be424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:45.424417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d296f-654e-401c-8ad2-3ec4f96be424 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:45.428358 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3175ba-5abe-4c53-ae85-f2d98e59ffef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:45.429019 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3175ba-5abe-4c53-ae85-f2d98e59ffef service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:45.429323 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3175ba-5abe-4c53-ae85-f2d98e59ffef service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:45.429586 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3175ba-5abe-4c53-ae85-f2d98e59ffef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:45.440224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3175ba-5abe-4c53-ae85-f2d98e59ffef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:45.441678 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3175ba-5abe-4c53-ae85-f2d98e59ffef service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:45.452506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1066/2139] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:45.772897 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a0ea6e-839f-4b79-b78c-8b5cae4508b2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:45.775927 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-83a0ea6e-839f-4b79-b78c-8b5cae4508b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:45.813709 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a0ea6e-839f-4b79-b78c-8b5cae4508b2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:45.823169 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61f8e0-f072-459a-b820-cc93f52a4b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:45.825050 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61f8e0-f072-459a-b820-cc93f52a4b94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:45.825757 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61f8e0-f072-459a-b820-cc93f52a4b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:45.826197 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61f8e0-f072-459a-b820-cc93f52a4b94 admin admin] PATH_INFO: `/v3/projects/cb5834cd36864dd9bb27f917f3841cd0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:45.827916 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e61f8e0-f072-459a-b820-cc93f52a4b94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb5834cd36864dd9bb27f917f3841cd0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:45.830729 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e61f8e0-f072-459a-b820-cc93f52a4b94 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:45.931134 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1e61f8e0-f072-459a-b820-cc93f52a4b94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb5834cd36864dd9bb27f917f3841cd0', 'request_id': 'req-1e61f8e0-f072-459a-b820-cc93f52a4b94'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:45.947181 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1e61f8e0-f072-459a-b820-cc93f52a4b94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb5834cd36864dd9bb27f917f3841cd0', 'request_id': 'req-1e61f8e0-f072-459a-b820-cc93f52a4b94'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:45.955565 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1e61f8e0-f072-459a-b820-cc93f52a4b94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb5834cd36864dd9bb27f917f3841cd0', 'request_id': 'req-1e61f8e0-f072-459a-b820-cc93f52a4b94'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:45.968797 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1e61f8e0-f072-459a-b820-cc93f52a4b94 admin admin] The token cache is being invalidate because project cb5834cd36864dd9bb27f917f3841cd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:45.969527 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1e61f8e0-f072-459a-b820-cc93f52a4b94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:45.971690 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1e61f8e0-f072-459a-b820-cc93f52a4b94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:45.973508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1074/2140] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:45 2022] DELETE /identity/v3/projects/cb5834cd36864dd9bb27f917f3841cd0 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:45.978370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3175ba-5abe-4c53-ae85-f2d98e59ffef service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:45.992785 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3175ba-5abe-4c53-ae85-f2d98e59ffef service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:46.199709 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fbc9e0-4915-4a93-8094-8aa1829f0e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:46.200618 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fbc9e0-4915-4a93-8094-8aa1829f0e41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:46.200867 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fbc9e0-4915-4a93-8094-8aa1829f0e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:46.201143 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fbc9e0-4915-4a93-8094-8aa1829f0e41 admin admin] PATH_INFO: `/v3/users/7186b29af5c7494a891b303887b27742` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:46.203290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fbc9e0-4915-4a93-8094-8aa1829f0e41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7186b29af5c7494a891b303887b27742)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:46.206915 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fbc9e0-4915-4a93-8094-8aa1829f0e41 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:46.296291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-42fbc9e0-4915-4a93-8094-8aa1829f0e41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7186b29af5c7494a891b303887b27742'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:46.306113 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-42fbc9e0-4915-4a93-8094-8aa1829f0e41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7186b29af5c7494a891b303887b27742'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:46.326452 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-42fbc9e0-4915-4a93-8094-8aa1829f0e41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7186b29af5c7494a891b303887b27742'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:46.327803 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-42fbc9e0-4915-4a93-8094-8aa1829f0e41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7186b29af5c7494a891b303887b27742'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:46.329431 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-42fbc9e0-4915-4a93-8094-8aa1829f0e41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7186b29af5c7494a891b303887b27742'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:46.339913 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:46.366532 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1067/2141] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:34:45 2022] DELETE /identity/v3/users/7186b29af5c7494a891b303887b27742 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:34:46.751006 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61f8e0-f072-459a-b820-cc93f52a4b94 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:46.754492 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1e61f8e0-f072-459a-b820-cc93f52a4b94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:46.785913 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61f8e0-f072-459a-b820-cc93f52a4b94 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:46.933308 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff3460-f7e5-4b77-bd61-365912ec0440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:46.935141 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff3460-f7e5-4b77-bd61-365912ec0440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:46.935632 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff3460-f7e5-4b77-bd61-365912ec0440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:46.935968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ff3460-f7e5-4b77-bd61-365912ec0440 admin admin] PATH_INFO: `/v3/projects/159edb8bb6b44df5a4dc88e0bdeab421` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:46.939119 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ff3460-f7e5-4b77-bd61-365912ec0440 admin admin] RBAC: Authorizing `identity:delete_project(project_id=159edb8bb6b44df5a4dc88e0bdeab421)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:46.941669 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ff3460-f7e5-4b77-bd61-365912ec0440 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:47.004225 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-14ff3460-f7e5-4b77-bd61-365912ec0440 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '159edb8bb6b44df5a4dc88e0bdeab421', 'request_id': 'req-14ff3460-f7e5-4b77-bd61-365912ec0440'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:47.013150 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-14ff3460-f7e5-4b77-bd61-365912ec0440 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '159edb8bb6b44df5a4dc88e0bdeab421', 'request_id': 'req-14ff3460-f7e5-4b77-bd61-365912ec0440'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:47.020874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-14ff3460-f7e5-4b77-bd61-365912ec0440 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '159edb8bb6b44df5a4dc88e0bdeab421', 'request_id': 'req-14ff3460-f7e5-4b77-bd61-365912ec0440'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:47.034092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-14ff3460-f7e5-4b77-bd61-365912ec0440 admin admin] The token cache is being invalidate because project 159edb8bb6b44df5a4dc88e0bdeab421 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:47.035010 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-14ff3460-f7e5-4b77-bd61-365912ec0440 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:47.035667 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-14ff3460-f7e5-4b77-bd61-365912ec0440 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:47.041385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1075/2142] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:46 2022] DELETE /identity/v3/projects/159edb8bb6b44df5a4dc88e0bdeab421 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:47.043215 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fbc9e0-4915-4a93-8094-8aa1829f0e41 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:47.045116 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-42fbc9e0-4915-4a93-8094-8aa1829f0e41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:47.061194 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fbc9e0-4915-4a93-8094-8aa1829f0e41 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:47.139234 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd9ed1-900d-40a7-9bbb-d39fa419f66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:47.140125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd9ed1-900d-40a7-9bbb-d39fa419f66a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:47.140415 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd9ed1-900d-40a7-9bbb-d39fa419f66a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:47.140698 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd9ed1-900d-40a7-9bbb-d39fa419f66a admin admin] PATH_INFO: `/v3/users/3104e8fd088040878d151b3b5dc42682` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:47.141921 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd9ed1-900d-40a7-9bbb-d39fa419f66a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3104e8fd088040878d151b3b5dc42682)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:47.143978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd9ed1-900d-40a7-9bbb-d39fa419f66a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:47.273458 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-64dd9ed1-900d-40a7-9bbb-d39fa419f66a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3104e8fd088040878d151b3b5dc42682'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:47.293612 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-64dd9ed1-900d-40a7-9bbb-d39fa419f66a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3104e8fd088040878d151b3b5dc42682'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:47.318363 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-64dd9ed1-900d-40a7-9bbb-d39fa419f66a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3104e8fd088040878d151b3b5dc42682'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:47.319853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-64dd9ed1-900d-40a7-9bbb-d39fa419f66a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3104e8fd088040878d151b3b5dc42682'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:47.320502 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-64dd9ed1-900d-40a7-9bbb-d39fa419f66a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3104e8fd088040878d151b3b5dc42682'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:47.343135 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1068/2143] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:34:47 2022] DELETE /identity/v3/users/3104e8fd088040878d151b3b5dc42682 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:34:48.058575 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff3460-f7e5-4b77-bd61-365912ec0440 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:48.061430 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-14ff3460-f7e5-4b77-bd61-365912ec0440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:48.091950 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:48.097939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ff3460-f7e5-4b77-bd61-365912ec0440 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:48.178720 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980a697-0c31-4245-9014-023dff7e2830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:48.179612 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980a697-0c31-4245-9014-023dff7e2830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:48.179910 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980a697-0c31-4245-9014-023dff7e2830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:48.180193 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980a697-0c31-4245-9014-023dff7e2830 admin admin] PATH_INFO: `/v3/projects/21353f0266d84c96af54e3778c63c77d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:48.181445 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980a697-0c31-4245-9014-023dff7e2830 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21353f0266d84c96af54e3778c63c77d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:48.183414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980a697-0c31-4245-9014-023dff7e2830 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:48.236564 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6980a697-0c31-4245-9014-023dff7e2830 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21353f0266d84c96af54e3778c63c77d', 'request_id': 'req-6980a697-0c31-4245-9014-023dff7e2830'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:48.248425 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6980a697-0c31-4245-9014-023dff7e2830 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21353f0266d84c96af54e3778c63c77d', 'request_id': 'req-6980a697-0c31-4245-9014-023dff7e2830'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:48.256114 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6980a697-0c31-4245-9014-023dff7e2830 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21353f0266d84c96af54e3778c63c77d', 'request_id': 'req-6980a697-0c31-4245-9014-023dff7e2830'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:48.266679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6980a697-0c31-4245-9014-023dff7e2830 admin admin] The token cache is being invalidate because project 21353f0266d84c96af54e3778c63c77d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:48.267056 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6980a697-0c31-4245-9014-023dff7e2830 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:48.268144 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6980a697-0c31-4245-9014-023dff7e2830 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:48.269588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1076/2144] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:48 2022] DELETE /identity/v3/projects/21353f0266d84c96af54e3778c63c77d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:48.274756 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112edfbc-c25f-4af7-84ff-87c0dda1aa94 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:48.275251 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112edfbc-c25f-4af7-84ff-87c0dda1aa94 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:48.275344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112edfbc-c25f-4af7-84ff-87c0dda1aa94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:48.293578 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-112edfbc-c25f-4af7-84ff-87c0dda1aa94 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:48.294778 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-112edfbc-c25f-4af7-84ff-87c0dda1aa94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:48.370211 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1069/2145] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:48.379581 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d465852-4139-4610-8e67-4121c578b60b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:48.379906 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d465852-4139-4610-8e67-4121c578b60b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:48.380195 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d465852-4139-4610-8e67-4121c578b60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:48.399768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8d465852-4139-4610-8e67-4121c578b60b None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:48.400996 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8d465852-4139-4610-8e67-4121c578b60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:48.414703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1077/2146] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:34:48.416415 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112edfbc-c25f-4af7-84ff-87c0dda1aa94 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:48.426576 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112edfbc-c25f-4af7-84ff-87c0dda1aa94 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:48.431605 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acdff5c-f919-438c-b4c6-96d2f1afa2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:48.432280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acdff5c-f919-438c-b4c6-96d2f1afa2b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:48.432570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acdff5c-f919-438c-b4c6-96d2f1afa2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:48.432853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acdff5c-f919-438c-b4c6-96d2f1afa2b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:48.433599 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acdff5c-f919-438c-b4c6-96d2f1afa2b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:34:48.433915 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acdff5c-f919-438c-b4c6-96d2f1afa2b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:48.435585 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acdff5c-f919-438c-b4c6-96d2f1afa2b3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:48.443835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:48.452989 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1070/2147] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:34:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:34:48.458831 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d465852-4139-4610-8e67-4121c578b60b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:48.470254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d465852-4139-4610-8e67-4121c578b60b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:48.479802 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c82c04-f063-4329-be50-c5277189d210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:48.480809 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c82c04-f063-4329-be50-c5277189d210 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:48.481483 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c82c04-f063-4329-be50-c5277189d210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:48.482183 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c82c04-f063-4329-be50-c5277189d210 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:48.483095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c82c04-f063-4329-be50-c5277189d210 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:48.485968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c82c04-f063-4329-be50-c5277189d210 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:48.528978 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1078/2148] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:34:48 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:48.530226 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acdff5c-f919-438c-b4c6-96d2f1afa2b3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:48.546246 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acdff5c-f919-438c-b4c6-96d2f1afa2b3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:48.696356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e73528-88c9-483d-9d49-4aca74a50fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:48.697155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e73528-88c9-483d-9d49-4aca74a50fde admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:48.697990 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e73528-88c9-483d-9d49-4aca74a50fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:48.698301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e73528-88c9-483d-9d49-4aca74a50fde admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:48.700786 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e73528-88c9-483d-9d49-4aca74a50fde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:48.715581 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e73528-88c9-483d-9d49-4aca74a50fde admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:48.755805 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1071/2149] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:34:48 2022] POST /identity/v3/users => generated 287 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:48.759534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c82c04-f063-4329-be50-c5277189d210 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:48.781594 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c82c04-f063-4329-be50-c5277189d210 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:48.788280 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625cec5-cfc7-4874-8d6d-e35092e585d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:48.788971 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625cec5-cfc7-4874-8d6d-e35092e585d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:48.789261 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625cec5-cfc7-4874-8d6d-e35092e585d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:48.789538 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7625cec5-cfc7-4874-8d6d-e35092e585d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:48.790285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625cec5-cfc7-4874-8d6d-e35092e585d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:48.792493 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7625cec5-cfc7-4874-8d6d-e35092e585d6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:48.820674 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1079/2150] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:48.822168 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e73528-88c9-483d-9d49-4aca74a50fde admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:48.838562 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e73528-88c9-483d-9d49-4aca74a50fde admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:48.853296 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6c38f-0eae-4180-bbb8-1a921d0eb69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:48.855831 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6c38f-0eae-4180-bbb8-1a921d0eb69c admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:48.857289 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6c38f-0eae-4180-bbb8-1a921d0eb69c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:48.858310 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6c38f-0eae-4180-bbb8-1a921d0eb69c admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:48.860971 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd6c38f-0eae-4180-bbb8-1a921d0eb69c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:48.868587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd6c38f-0eae-4180-bbb8-1a921d0eb69c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:48.897708 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1072/2151] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:48 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:34:48.902503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625cec5-cfc7-4874-8d6d-e35092e585d6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:48.939856 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7625cec5-cfc7-4874-8d6d-e35092e585d6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:48.948966 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ae2eb-74b4-42cc-80a2-23c3d134e23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:48.950413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ae2eb-74b4-42cc-80a2-23c3d134e23a admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:48.951482 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ae2eb-74b4-42cc-80a2-23c3d134e23a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:48.952035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ae2eb-74b4-42cc-80a2-23c3d134e23a admin admin] PATH_INFO: `/v3/projects/cc42fd82d68742f8a3a44125d93ca64a/users/26bafc33f08a4a97b9706a9c4e391615/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:49.005402 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ae2eb-74b4-42cc-80a2-23c3d134e23a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc42fd82d68742f8a3a44125d93ca64a, user_id=26bafc33f08a4a97b9706a9c4e391615, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:49.008493 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ae2eb-74b4-42cc-80a2-23c3d134e23a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:49.042615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1080/2152] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:48 2022] PUT /identity/v3/projects/cc42fd82d68742f8a3a44125d93ca64a/users/26bafc33f08a4a97b9706a9c4e391615/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:49.055603 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf58f81-c8b8-426a-a276-5311ec747da1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:49.056139 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf58f81-c8b8-426a-a276-5311ec747da1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:49.056676 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf58f81-c8b8-426a-a276-5311ec747da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:49.123767 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-caf58f81-c8b8-426a-a276-5311ec747da1 None None] MFA Rules not processed for user `26bafc33f08a4a97b9706a9c4e391615`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:49.125077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-caf58f81-c8b8-426a-a276-5311ec747da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:49.256126 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1073/2153] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:49.257441 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ae2eb-74b4-42cc-80a2-23c3d134e23a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:49.259422 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-115ae2eb-74b4-42cc-80a2-23c3d134e23a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:49.298845 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:49.332792 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf58f81-c8b8-426a-a276-5311ec747da1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:49.334797 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-caf58f81-c8b8-426a-a276-5311ec747da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:49.363564 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ae2eb-74b4-42cc-80a2-23c3d134e23a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:49.374886 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be473d94-8e1b-416c-8125-b92181b6eda4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:49.376246 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be473d94-8e1b-416c-8125-b92181b6eda4 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:49.378736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be473d94-8e1b-416c-8125-b92181b6eda4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:49.380595 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be473d94-8e1b-416c-8125-b92181b6eda4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:49.398156 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be473d94-8e1b-416c-8125-b92181b6eda4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:49.399967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be473d94-8e1b-416c-8125-b92181b6eda4 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:49.411088 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1081/2154] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:49.417178 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf58f81-c8b8-426a-a276-5311ec747da1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:49.422927 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac00367f-6c11-47ff-b558-9358c57fbbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:49.424102 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00367f-6c11-47ff-b558-9358c57fbbae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:49.424844 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00367f-6c11-47ff-b558-9358c57fbbae admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:49.424999 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00367f-6c11-47ff-b558-9358c57fbbae admin admin] PATH_INFO: `/v3/users/8a232c3292ec4a0abda3a8bc91c5a267` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:49.426289 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac00367f-6c11-47ff-b558-9358c57fbbae admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a232c3292ec4a0abda3a8bc91c5a267)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:49.428331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac00367f-6c11-47ff-b558-9358c57fbbae admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:49.437263 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:49.446493 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be473d94-8e1b-416c-8125-b92181b6eda4 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:49.465417 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be473d94-8e1b-416c-8125-b92181b6eda4 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:49.471573 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014c030-b134-492a-920c-571c06317bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:49.473724 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014c030-b134-492a-920c-571c06317bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:49.474480 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014c030-b134-492a-920c-571c06317bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:49.476684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014c030-b134-492a-920c-571c06317bba admin admin] PATH_INFO: `/v3/users/26bafc33f08a4a97b9706a9c4e391615` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:49.476684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014c030-b134-492a-920c-571c06317bba admin admin] RBAC: Authorizing `identity:delete_user(user_id=26bafc33f08a4a97b9706a9c4e391615)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:49.478516 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014c030-b134-492a-920c-571c06317bba admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:49.503522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ac00367f-6c11-47ff-b558-9358c57fbbae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a232c3292ec4a0abda3a8bc91c5a267'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:49.515892 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ac00367f-6c11-47ff-b558-9358c57fbbae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a232c3292ec4a0abda3a8bc91c5a267'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:49.532400 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ac00367f-6c11-47ff-b558-9358c57fbbae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a232c3292ec4a0abda3a8bc91c5a267'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:49.533826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ac00367f-6c11-47ff-b558-9358c57fbbae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a232c3292ec4a0abda3a8bc91c5a267'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:49.534602 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ac00367f-6c11-47ff-b558-9358c57fbbae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a232c3292ec4a0abda3a8bc91c5a267'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:49.544062 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a014c030-b134-492a-920c-571c06317bba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26bafc33f08a4a97b9706a9c4e391615'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:49.544796 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a014c030-b134-492a-920c-571c06317bba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26bafc33f08a4a97b9706a9c4e391615'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:49.547826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1074/2155] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:34:49 2022] DELETE /identity/v3/users/8a232c3292ec4a0abda3a8bc91c5a267 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:49.558062 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a014c030-b134-492a-920c-571c06317bba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26bafc33f08a4a97b9706a9c4e391615'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:49.558888 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a014c030-b134-492a-920c-571c06317bba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26bafc33f08a4a97b9706a9c4e391615'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:49.565948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a014c030-b134-492a-920c-571c06317bba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26bafc33f08a4a97b9706a9c4e391615'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:49.581481 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1082/2156] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:34:49 2022] DELETE /identity/v3/users/26bafc33f08a4a97b9706a9c4e391615 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:49.586116 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99266175-5a65-4cbb-8650-c2f4ebed1775 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:49.586528 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99266175-5a65-4cbb-8650-c2f4ebed1775 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:49.586888 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99266175-5a65-4cbb-8650-c2f4ebed1775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:49.608082 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-99266175-5a65-4cbb-8650-c2f4ebed1775 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:49.609163 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-99266175-5a65-4cbb-8650-c2f4ebed1775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:49.672660 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1075/2157] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:49.683425 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014c030-b134-492a-920c-571c06317bba admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:49.684880 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a014c030-b134-492a-920c-571c06317bba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:49.738190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99266175-5a65-4cbb-8650-c2f4ebed1775 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:49.739728 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-99266175-5a65-4cbb-8650-c2f4ebed1775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:49.752974 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99266175-5a65-4cbb-8650-c2f4ebed1775 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:49.760330 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014c030-b134-492a-920c-571c06317bba admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:49.765482 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c65f1-9723-4f1d-8cc2-916fdacc6414 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:49.766082 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c65f1-9723-4f1d-8cc2-916fdacc6414 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:49.766456 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c65f1-9723-4f1d-8cc2-916fdacc6414 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:49.766855 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c65f1-9723-4f1d-8cc2-916fdacc6414 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:49.769238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f74f3c4-81df-4baf-8b5e-5d8d3cd43711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:49.770104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74f3c4-81df-4baf-8b5e-5d8d3cd43711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:49.770701 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74f3c4-81df-4baf-8b5e-5d8d3cd43711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:49.770935 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f74f3c4-81df-4baf-8b5e-5d8d3cd43711 admin admin] PATH_INFO: `/v3/projects/46f5397c062a4868a0282f7535f84f7a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:49.772153 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f74f3c4-81df-4baf-8b5e-5d8d3cd43711 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46f5397c062a4868a0282f7535f84f7a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:49.774133 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f74f3c4-81df-4baf-8b5e-5d8d3cd43711 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:49.778485 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c65f1-9723-4f1d-8cc2-916fdacc6414 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:49.780361 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c65f1-9723-4f1d-8cc2-916fdacc6414 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:49.793099 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1083/2158] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:49.823471 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7f74f3c4-81df-4baf-8b5e-5d8d3cd43711 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46f5397c062a4868a0282f7535f84f7a', 'request_id': 'req-7f74f3c4-81df-4baf-8b5e-5d8d3cd43711'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:49.830715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7f74f3c4-81df-4baf-8b5e-5d8d3cd43711 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46f5397c062a4868a0282f7535f84f7a', 'request_id': 'req-7f74f3c4-81df-4baf-8b5e-5d8d3cd43711'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:49.837103 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7f74f3c4-81df-4baf-8b5e-5d8d3cd43711 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46f5397c062a4868a0282f7535f84f7a', 'request_id': 'req-7f74f3c4-81df-4baf-8b5e-5d8d3cd43711'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:49.850863 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7f74f3c4-81df-4baf-8b5e-5d8d3cd43711 admin admin] The token cache is being invalidate because project 46f5397c062a4868a0282f7535f84f7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:49.851203 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7f74f3c4-81df-4baf-8b5e-5d8d3cd43711 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:49.851966 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7f74f3c4-81df-4baf-8b5e-5d8d3cd43711 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:49.853870 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1076/2159] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:49 2022] DELETE /identity/v3/projects/46f5397c062a4868a0282f7535f84f7a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:49.859487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917bd12-b1d4-4107-9138-985eac02a9c0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:49.859851 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917bd12-b1d4-4107-9138-985eac02a9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:49.860306 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3917bd12-b1d4-4107-9138-985eac02a9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:49.879850 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3917bd12-b1d4-4107-9138-985eac02a9c0 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:49.881086 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3917bd12-b1d4-4107-9138-985eac02a9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:49.931583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:49.954058 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1084/2160] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:49.963819 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51d958-e654-4df1-818b-83d4b7a0eb61 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:49.964004 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51d958-e654-4df1-818b-83d4b7a0eb61 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:49.964297 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51d958-e654-4df1-818b-83d4b7a0eb61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:49.984397 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5e51d958-e654-4df1-818b-83d4b7a0eb61 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:49.985606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5e51d958-e654-4df1-818b-83d4b7a0eb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:49.994480 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1077/2161] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:49.999031 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917bd12-b1d4-4107-9138-985eac02a9c0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:50.009570 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3917bd12-b1d4-4107-9138-985eac02a9c0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:50.014541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf9c92-2b90-4be1-a740-2d12a42c3143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:50.015209 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf9c92-2b90-4be1-a740-2d12a42c3143 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:50.015494 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf9c92-2b90-4be1-a740-2d12a42c3143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:50.015799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf9c92-2b90-4be1-a740-2d12a42c3143 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:50.016670 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf9c92-2b90-4be1-a740-2d12a42c3143 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:34:50.016980 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf9c92-2b90-4be1-a740-2d12a42c3143 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:50.018717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bf9c92-2b90-4be1-a740-2d12a42c3143 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:50.031695 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1085/2162] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:34:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:34:50.036684 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51d958-e654-4df1-818b-83d4b7a0eb61 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:50.041142 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:50.047496 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51d958-e654-4df1-818b-83d4b7a0eb61 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:50.053495 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0ca3c-8606-49b2-a884-f757e532d914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:50.054257 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0ca3c-8606-49b2-a884-f757e532d914 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:50.054659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0ca3c-8606-49b2-a884-f757e532d914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:50.055012 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0ca3c-8606-49b2-a884-f757e532d914 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:50.055771 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0ca3c-8606-49b2-a884-f757e532d914 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:50.057751 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0ca3c-8606-49b2-a884-f757e532d914 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:50.086116 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1078/2163] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:34:50 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:50.090335 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf9c92-2b90-4be1-a740-2d12a42c3143 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:50.101756 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bf9c92-2b90-4be1-a740-2d12a42c3143 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:50.191495 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c0457-c963-461f-9e22-4e0f3a79f992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:50.192500 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0ca3c-8606-49b2-a884-f757e532d914 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:50.192913 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c0457-c963-461f-9e22-4e0f3a79f992 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:50.194319 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0bf0ca3c-8606-49b2-a884-f757e532d914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:50.196854 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c0457-c963-461f-9e22-4e0f3a79f992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:50.197160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c0457-c963-461f-9e22-4e0f3a79f992 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:50.198047 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c0457-c963-461f-9e22-4e0f3a79f992 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:50.200078 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213c0457-c963-461f-9e22-4e0f3a79f992 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:50.213349 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0ca3c-8606-49b2-a884-f757e532d914 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:50.219809 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f5d72-ca59-49f4-966d-a852e51934e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:50.221357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f5d72-ca59-49f4-966d-a852e51934e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:50.222259 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f5d72-ca59-49f4-966d-a852e51934e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:50.222259 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f5d72-ca59-49f4-966d-a852e51934e7 admin admin] PATH_INFO: `/v3/projects/cc42fd82d68742f8a3a44125d93ca64a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:50.223758 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f5d72-ca59-49f4-966d-a852e51934e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc42fd82d68742f8a3a44125d93ca64a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:50.229854 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368f5d72-ca59-49f4-966d-a852e51934e7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:50.232189 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1086/2164] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:34:50 2022] POST /identity/v3/users => generated 292 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:50.233198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c0457-c963-461f-9e22-4e0f3a79f992 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:50.244640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c0457-c963-461f-9e22-4e0f3a79f992 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:50.249556 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003814b6-df1f-4fc0-a59d-686bb932da55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:50.250228 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003814b6-df1f-4fc0-a59d-686bb932da55 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:50.250558 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003814b6-df1f-4fc0-a59d-686bb932da55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:50.250841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003814b6-df1f-4fc0-a59d-686bb932da55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:50.251569 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003814b6-df1f-4fc0-a59d-686bb932da55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:50.253931 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003814b6-df1f-4fc0-a59d-686bb932da55 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:50.264632 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1087/2165] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:50.269606 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003814b6-df1f-4fc0-a59d-686bb932da55 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:50.280863 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003814b6-df1f-4fc0-a59d-686bb932da55 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:50.294098 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-368f5d72-ca59-49f4-966d-a852e51934e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc42fd82d68742f8a3a44125d93ca64a', 'request_id': 'req-368f5d72-ca59-49f4-966d-a852e51934e7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:50.306913 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-368f5d72-ca59-49f4-966d-a852e51934e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc42fd82d68742f8a3a44125d93ca64a', 'request_id': 'req-368f5d72-ca59-49f4-966d-a852e51934e7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:50.320050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-368f5d72-ca59-49f4-966d-a852e51934e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc42fd82d68742f8a3a44125d93ca64a', 'request_id': 'req-368f5d72-ca59-49f4-966d-a852e51934e7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:50.340823 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-368f5d72-ca59-49f4-966d-a852e51934e7 admin admin] The token cache is being invalidate because project cc42fd82d68742f8a3a44125d93ca64a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:50.341346 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-368f5d72-ca59-49f4-966d-a852e51934e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:50.343120 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-368f5d72-ca59-49f4-966d-a852e51934e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:50.345518 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1079/2166] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:50 2022] DELETE /identity/v3/projects/cc42fd82d68742f8a3a44125d93ca64a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:34:50.350626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebf81f-089e-40a9-9bc3-97b60419ad7d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:50.351006 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebf81f-089e-40a9-9bc3-97b60419ad7d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:50.351353 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ebf81f-089e-40a9-9bc3-97b60419ad7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:50.368948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2934f6-59cf-4063-95be-3258d58569f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:50.369648 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2934f6-59cf-4063-95be-3258d58569f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:50.369953 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2934f6-59cf-4063-95be-3258d58569f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:50.370057 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-35ebf81f-089e-40a9-9bc3-97b60419ad7d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:50.370221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2934f6-59cf-4063-95be-3258d58569f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:50.370998 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2934f6-59cf-4063-95be-3258d58569f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:50.371398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-35ebf81f-089e-40a9-9bc3-97b60419ad7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:50.372608 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2934f6-59cf-4063-95be-3258d58569f5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:50.390370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1080/2167] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:34:50.397579 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1088/2168] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:50.399572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ebf81f-089e-40a9-9bc3-97b60419ad7d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:50.400763 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf515a3-4c27-4511-abd7-5123087969fc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:50.401034 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-35ebf81f-089e-40a9-9bc3-97b60419ad7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:50.401160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf515a3-4c27-4511-abd7-5123087969fc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:50.401340 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf515a3-4c27-4511-abd7-5123087969fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:50.416011 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ebf81f-089e-40a9-9bc3-97b60419ad7d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:50.428752 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-daf515a3-4c27-4511-abd7-5123087969fc None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:50.429990 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-daf515a3-4c27-4511-abd7-5123087969fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:50.437776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a493d0-ab24-42b3-878b-cf1e2d05254b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:50.438913 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a493d0-ab24-42b3-878b-cf1e2d05254b admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:50.440134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a493d0-ab24-42b3-878b-cf1e2d05254b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:50.440134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a493d0-ab24-42b3-878b-cf1e2d05254b admin admin] PATH_INFO: `/v3/projects/cb6b59e079b84caba3c21bc469610368/users/0914a48fe85c4980b80b5bfc2ed8cca3/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:50.443360 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1089/2169] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:50.444419 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf515a3-4c27-4511-abd7-5123087969fc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:50.453496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf515a3-4c27-4511-abd7-5123087969fc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:50.458949 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67cccd-5756-4568-9586-cad72b276943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:50.459623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67cccd-5756-4568-9586-cad72b276943 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:50.459920 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67cccd-5756-4568-9586-cad72b276943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:50.460118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a493d0-ab24-42b3-878b-cf1e2d05254b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb6b59e079b84caba3c21bc469610368, user_id=0914a48fe85c4980b80b5bfc2ed8cca3, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:50.460251 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67cccd-5756-4568-9586-cad72b276943 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:50.460945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67cccd-5756-4568-9586-cad72b276943 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:34:50.461258 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67cccd-5756-4568-9586-cad72b276943 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:50.462818 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a493d0-ab24-42b3-878b-cf1e2d05254b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:50.462938 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67cccd-5756-4568-9586-cad72b276943 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:50.476925 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1090/2170] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:34:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:34:50.480347 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1081/2171] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:50 2022] PUT /identity/v3/projects/cb6b59e079b84caba3c21bc469610368/users/0914a48fe85c4980b80b5bfc2ed8cca3/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:50.483448 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67cccd-5756-4568-9586-cad72b276943 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:50.490317 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61dfd0-89bd-40ee-926c-40cb94830657 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:50.490634 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61dfd0-89bd-40ee-926c-40cb94830657 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:50.490944 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61dfd0-89bd-40ee-926c-40cb94830657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:50.493594 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67cccd-5756-4568-9586-cad72b276943 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:50.519092 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ee61dfd0-89bd-40ee-926c-40cb94830657 None None] MFA Rules not processed for user `0914a48fe85c4980b80b5bfc2ed8cca3`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:50.520796 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ee61dfd0-89bd-40ee-926c-40cb94830657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:50.527177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:50.557470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f13a3d-13aa-47a2-b6ef-42d1940dd43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:50.558154 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f13a3d-13aa-47a2-b6ef-42d1940dd43a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:50.558463 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f13a3d-13aa-47a2-b6ef-42d1940dd43a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:50.558751 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f13a3d-13aa-47a2-b6ef-42d1940dd43a admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:50.560224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f13a3d-13aa-47a2-b6ef-42d1940dd43a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:50.562072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f13a3d-13aa-47a2-b6ef-42d1940dd43a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:50.593629 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1082/2172] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:34:50.598402 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1091/2173] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:34:50 2022] POST /identity/v3/projects => generated 399 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:50.598801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee61dfd0-89bd-40ee-926c-40cb94830657 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:50.601995 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f13a3d-13aa-47a2-b6ef-42d1940dd43a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:50.609994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee61dfd0-89bd-40ee-926c-40cb94830657 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:50.632955 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:50.677164 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f13a3d-13aa-47a2-b6ef-42d1940dd43a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:50.680080 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07caa39c-23c8-40fe-ac25-4137682590d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:50.680809 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07caa39c-23c8-40fe-ac25-4137682590d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:50.681115 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07caa39c-23c8-40fe-ac25-4137682590d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:50.681397 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07caa39c-23c8-40fe-ac25-4137682590d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:50.681857 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15297e58-124c-4d70-8a13-d50fbf40cb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:50.682128 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07caa39c-23c8-40fe-ac25-4137682590d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:50.682602 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15297e58-124c-4d70-8a13-d50fbf40cb71 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:50.682900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15297e58-124c-4d70-8a13-d50fbf40cb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:50.683187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15297e58-124c-4d70-8a13-d50fbf40cb71 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:50.683880 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15297e58-124c-4d70-8a13-d50fbf40cb71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:50.684413 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07caa39c-23c8-40fe-ac25-4137682590d4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:50.685794 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15297e58-124c-4d70-8a13-d50fbf40cb71 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:50.707284 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1092/2174] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:34:50 2022] POST /identity/v3/users => generated 300 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:34:50.713907 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15297e58-124c-4d70-8a13-d50fbf40cb71 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:50.714050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1083/2175] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:34:50 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:50.714953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07caa39c-23c8-40fe-ac25-4137682590d4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:50.721630 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15297e58-124c-4d70-8a13-d50fbf40cb71 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:50.781342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c033a2-6a20-4e82-9d03-3830260ec0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:50.781830 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07caa39c-23c8-40fe-ac25-4137682590d4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:50.782018 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c033a2-6a20-4e82-9d03-3830260ec0d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:50.782307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c033a2-6a20-4e82-9d03-3830260ec0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:50.782626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c033a2-6a20-4e82-9d03-3830260ec0d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:50.783906 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c033a2-6a20-4e82-9d03-3830260ec0d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:50.785615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c033a2-6a20-4e82-9d03-3830260ec0d1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:50.788541 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271c7ea-fdee-4d6c-8c6c-81f53586dcb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:50.789273 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271c7ea-fdee-4d6c-8c6c-81f53586dcb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:50.789517 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271c7ea-fdee-4d6c-8c6c-81f53586dcb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:50.789800 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271c7ea-fdee-4d6c-8c6c-81f53586dcb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:50.790548 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271c7ea-fdee-4d6c-8c6c-81f53586dcb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:50.792515 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271c7ea-fdee-4d6c-8c6c-81f53586dcb8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:50.795923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1093/2176] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:34:50.800545 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c033a2-6a20-4e82-9d03-3830260ec0d1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:50.810320 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c033a2-6a20-4e82-9d03-3830260ec0d1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:50.813801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1084/2177] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:34:50 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:34:50.815598 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604e888-72b8-4bf9-b413-7d62929f5682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:50.816263 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604e888-72b8-4bf9-b413-7d62929f5682 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:50.816545 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604e888-72b8-4bf9-b413-7d62929f5682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:50.816825 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604e888-72b8-4bf9-b413-7d62929f5682 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:50.817620 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5604e888-72b8-4bf9-b413-7d62929f5682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:50.817924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271c7ea-fdee-4d6c-8c6c-81f53586dcb8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:50.819222 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5604e888-72b8-4bf9-b413-7d62929f5682 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:50.828072 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271c7ea-fdee-4d6c-8c6c-81f53586dcb8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:50.829909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1094/2178] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:34:50.833184 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aff96a-4523-4804-b6b4-8d5ea3e817ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:50.833855 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aff96a-4523-4804-b6b4-8d5ea3e817ec admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:50.834144 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aff96a-4523-4804-b6b4-8d5ea3e817ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:50.834304 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604e888-72b8-4bf9-b413-7d62929f5682 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:50.834410 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aff96a-4523-4804-b6b4-8d5ea3e817ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:50.835170 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aff96a-4523-4804-b6b4-8d5ea3e817ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:50.836804 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aff96a-4523-4804-b6b4-8d5ea3e817ec admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:50.843934 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604e888-72b8-4bf9-b413-7d62929f5682 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:50.848568 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1085/2179] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:34:50.849458 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eafcf4-fd56-4214-bd73-b8c5b9f3f23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:50.850522 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eafcf4-fd56-4214-bd73-b8c5b9f3f23a admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:50.850811 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eafcf4-fd56-4214-bd73-b8c5b9f3f23a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:50.851168 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eafcf4-fd56-4214-bd73-b8c5b9f3f23a admin admin] PATH_INFO: `/v3/projects/b51e48337ed549918fec2c24595597f0/users/83a783c80c1c401d872ef22e6b461938/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:50.852951 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aff96a-4523-4804-b6b4-8d5ea3e817ec admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:50.864107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aff96a-4523-4804-b6b4-8d5ea3e817ec admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:50.867024 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eafcf4-fd56-4214-bd73-b8c5b9f3f23a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b51e48337ed549918fec2c24595597f0, user_id=83a783c80c1c401d872ef22e6b461938, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:50.868936 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388dd6ec-b362-454a-9b93-727e41d00c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:50.869694 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eafcf4-fd56-4214-bd73-b8c5b9f3f23a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:50.869798 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388dd6ec-b362-454a-9b93-727e41d00c73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:50.870085 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388dd6ec-b362-454a-9b93-727e41d00c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:50.870364 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388dd6ec-b362-454a-9b93-727e41d00c73 admin admin] PATH_INFO: `/v3/projects/0f0f92e91f6a432185c15b60980bd11e/users/f6c81addfec54d0f88266e667a3d3315/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:50.884068 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1095/2180] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:50 2022] PUT /identity/v3/projects/b51e48337ed549918fec2c24595597f0/users/83a783c80c1c401d872ef22e6b461938/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:34:50.887807 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388dd6ec-b362-454a-9b93-727e41d00c73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f0f92e91f6a432185c15b60980bd11e, user_id=f6c81addfec54d0f88266e667a3d3315, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:50.890492 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388dd6ec-b362-454a-9b93-727e41d00c73 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:50.895912 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648fbbef-adef-4a61-ab13-92b959312cd7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:50.896484 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648fbbef-adef-4a61-ab13-92b959312cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:50.896839 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648fbbef-adef-4a61-ab13-92b959312cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:50.905609 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1086/2181] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:50 2022] PUT /identity/v3/projects/0f0f92e91f6a432185c15b60980bd11e/users/f6c81addfec54d0f88266e667a3d3315/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:34:50.910864 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:50.911033 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388dd6ec-b362-454a-9b93-727e41d00c73 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:50.928022 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-648fbbef-adef-4a61-ab13-92b959312cd7 None None] MFA Rules not processed for user `83a783c80c1c401d872ef22e6b461938`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:50.929246 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-648fbbef-adef-4a61-ab13-92b959312cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:50.980423 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388dd6ec-b362-454a-9b93-727e41d00c73 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:50.988067 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff82950-a5e5-4cef-9fcf-5fbd8382cbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:50.989368 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff82950-a5e5-4cef-9fcf-5fbd8382cbdc admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:50.989368 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff82950-a5e5-4cef-9fcf-5fbd8382cbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:50.989587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff82950-a5e5-4cef-9fcf-5fbd8382cbdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:50.990304 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff82950-a5e5-4cef-9fcf-5fbd8382cbdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:50.993578 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff82950-a5e5-4cef-9fcf-5fbd8382cbdc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:50.997135 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:51.018463 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1087/2182] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:34:51.021518 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff82950-a5e5-4cef-9fcf-5fbd8382cbdc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:51.025383 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1096/2183] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:51.032672 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff82950-a5e5-4cef-9fcf-5fbd8382cbdc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:51.038436 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a1eafa-064c-41da-adc2-662256485faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:51.038754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648fbbef-adef-4a61-ab13-92b959312cd7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:51.039340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1eafa-064c-41da-adc2-662256485faf admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:51.039701 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1eafa-064c-41da-adc2-662256485faf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:51.040007 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a1eafa-064c-41da-adc2-662256485faf admin admin] PATH_INFO: `/v3/projects/0f0f92e91f6a432185c15b60980bd11e/users/f6c81addfec54d0f88266e667a3d3315/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:51.041277 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-648fbbef-adef-4a61-ab13-92b959312cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:51.042607 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1eafa-064c-41da-adc2-662256485faf admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f0f92e91f6a432185c15b60980bd11e, user_id=f6c81addfec54d0f88266e667a3d3315, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:51.045185 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a1eafa-064c-41da-adc2-662256485faf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:51.057958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1088/2184] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:51 2022] PUT /identity/v3/projects/0f0f92e91f6a432185c15b60980bd11e/users/f6c81addfec54d0f88266e667a3d3315/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:51.068362 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e88276-354e-45b2-9589-eddee58982ce None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:51.068785 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e88276-354e-45b2-9589-eddee58982ce None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:51.069143 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e88276-354e-45b2-9589-eddee58982ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:51.102328 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-25e88276-354e-45b2-9589-eddee58982ce None None] MFA Rules not processed for user `f6c81addfec54d0f88266e667a3d3315`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:51.103939 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-25e88276-354e-45b2-9589-eddee58982ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:51.114785 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648fbbef-adef-4a61-ab13-92b959312cd7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:51.181588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca96a4a-37be-442f-ac7c-130bd15293a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:51.182075 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca96a4a-37be-442f-ac7c-130bd15293a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:51.182368 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca96a4a-37be-442f-ac7c-130bd15293a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:51.182688 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca96a4a-37be-442f-ac7c-130bd15293a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:51.204371 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1089/2185] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:51.263688 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e88276-354e-45b2-9589-eddee58982ce None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:51.265247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-25e88276-354e-45b2-9589-eddee58982ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:51.272336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca96a4a-37be-442f-ac7c-130bd15293a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:51.274200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca96a4a-37be-442f-ac7c-130bd15293a7 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:51.281860 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e88276-354e-45b2-9589-eddee58982ce None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:51.285775 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1097/2186] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:51 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:51.287927 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ed3fd-b7b5-4f82-b5bf-c11f7231b73e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:51.288650 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ed3fd-b7b5-4f82-b5bf-c11f7231b73e service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:51.288935 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ed3fd-b7b5-4f82-b5bf-c11f7231b73e service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:51.289140 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ed3fd-b7b5-4f82-b5bf-c11f7231b73e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:51.304673 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310ed3fd-b7b5-4f82-b5bf-c11f7231b73e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:51.306162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310ed3fd-b7b5-4f82-b5bf-c11f7231b73e service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:51.321805 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1090/2187] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:51 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:51.366992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca96a4a-37be-442f-ac7c-130bd15293a7 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:51.385581 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:51.511465 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca96a4a-37be-442f-ac7c-130bd15293a7 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:51.519649 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29fba3-2714-4c7a-8574-bc7ad9305670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:51.521941 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29fba3-2714-4c7a-8574-bc7ad9305670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:51.522510 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29fba3-2714-4c7a-8574-bc7ad9305670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:51.522933 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29fba3-2714-4c7a-8574-bc7ad9305670 admin admin] PATH_INFO: `/v3/users/0914a48fe85c4980b80b5bfc2ed8cca3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:51.524972 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29fba3-2714-4c7a-8574-bc7ad9305670 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0914a48fe85c4980b80b5bfc2ed8cca3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:51.527118 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29fba3-2714-4c7a-8574-bc7ad9305670 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:51.589946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5e29fba3-2714-4c7a-8574-bc7ad9305670 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0914a48fe85c4980b80b5bfc2ed8cca3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:51.590850 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5e29fba3-2714-4c7a-8574-bc7ad9305670 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0914a48fe85c4980b80b5bfc2ed8cca3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:51.603936 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5e29fba3-2714-4c7a-8574-bc7ad9305670 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0914a48fe85c4980b80b5bfc2ed8cca3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:51.604484 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5e29fba3-2714-4c7a-8574-bc7ad9305670 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0914a48fe85c4980b80b5bfc2ed8cca3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:51.613568 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5e29fba3-2714-4c7a-8574-bc7ad9305670 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0914a48fe85c4980b80b5bfc2ed8cca3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:51.630589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1098/2188] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:34:51 2022] DELETE /identity/v3/users/0914a48fe85c4980b80b5bfc2ed8cca3 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:51.635596 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554f110-2de1-4440-9ae6-019e1be3e826 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:51.636140 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554f110-2de1-4440-9ae6-019e1be3e826 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:51.636471 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554f110-2de1-4440-9ae6-019e1be3e826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:51.664760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7554f110-2de1-4440-9ae6-019e1be3e826 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:51.665958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7554f110-2de1-4440-9ae6-019e1be3e826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:51.730531 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1091/2189] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:51.740939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29fba3-2714-4c7a-8574-bc7ad9305670 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:51.742887 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5e29fba3-2714-4c7a-8574-bc7ad9305670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:51.805553 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29fba3-2714-4c7a-8574-bc7ad9305670 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:51.809493 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce85a6e-bf0d-434c-9d52-358da079be21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:51.810132 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce85a6e-bf0d-434c-9d52-358da079be21 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:51.810456 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce85a6e-bf0d-434c-9d52-358da079be21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:51.810736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce85a6e-bf0d-434c-9d52-358da079be21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:51.822562 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce85a6e-bf0d-434c-9d52-358da079be21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:51.823408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce85a6e-bf0d-434c-9d52-358da079be21 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:51.835022 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1099/2190] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:52.141677 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554f110-2de1-4440-9ae6-019e1be3e826 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:52.143351 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7554f110-2de1-4440-9ae6-019e1be3e826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:52.155655 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554f110-2de1-4440-9ae6-019e1be3e826 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:52.161549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cea022-4c5a-4c6e-8a3b-9d432ffd78dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:52.162519 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cea022-4c5a-4c6e-8a3b-9d432ffd78dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:52.162875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cea022-4c5a-4c6e-8a3b-9d432ffd78dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:52.163221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cea022-4c5a-4c6e-8a3b-9d432ffd78dc admin admin] PATH_INFO: `/v3/projects/cb6b59e079b84caba3c21bc469610368` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:52.164717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cea022-4c5a-4c6e-8a3b-9d432ffd78dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb6b59e079b84caba3c21bc469610368)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:52.167089 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cea022-4c5a-4c6e-8a3b-9d432ffd78dc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:52.221699 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a1cea022-4c5a-4c6e-8a3b-9d432ffd78dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb6b59e079b84caba3c21bc469610368', 'request_id': 'req-a1cea022-4c5a-4c6e-8a3b-9d432ffd78dc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:52.223614 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:52.228401 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a1cea022-4c5a-4c6e-8a3b-9d432ffd78dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb6b59e079b84caba3c21bc469610368', 'request_id': 'req-a1cea022-4c5a-4c6e-8a3b-9d432ffd78dc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:52.235191 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a1cea022-4c5a-4c6e-8a3b-9d432ffd78dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb6b59e079b84caba3c21bc469610368', 'request_id': 'req-a1cea022-4c5a-4c6e-8a3b-9d432ffd78dc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:52.250147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a1cea022-4c5a-4c6e-8a3b-9d432ffd78dc admin admin] The token cache is being invalidate because project cb6b59e079b84caba3c21bc469610368 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:52.250576 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a1cea022-4c5a-4c6e-8a3b-9d432ffd78dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:52.251380 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a1cea022-4c5a-4c6e-8a3b-9d432ffd78dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:52.254077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1092/2191] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:52 2022] DELETE /identity/v3/projects/cb6b59e079b84caba3c21bc469610368 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:52.258340 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce85a6e-bf0d-434c-9d52-358da079be21 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:52.268972 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce85a6e-bf0d-434c-9d52-358da079be21 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:52.300923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:52.334828 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909cc15-6dfd-47d4-8d5e-093301c1382d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:52.335928 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909cc15-6dfd-47d4-8d5e-093301c1382d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:52.336216 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909cc15-6dfd-47d4-8d5e-093301c1382d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:52.336499 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909cc15-6dfd-47d4-8d5e-093301c1382d admin admin] PATH_INFO: `/v3/users/f6c81addfec54d0f88266e667a3d3315` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:52.338026 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b909cc15-6dfd-47d4-8d5e-093301c1382d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6c81addfec54d0f88266e667a3d3315)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:52.340238 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b909cc15-6dfd-47d4-8d5e-093301c1382d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:52.393086 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b909cc15-6dfd-47d4-8d5e-093301c1382d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6c81addfec54d0f88266e667a3d3315'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:52.393819 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b909cc15-6dfd-47d4-8d5e-093301c1382d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6c81addfec54d0f88266e667a3d3315'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:52.407525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b909cc15-6dfd-47d4-8d5e-093301c1382d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6c81addfec54d0f88266e667a3d3315'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:52.409278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b909cc15-6dfd-47d4-8d5e-093301c1382d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6c81addfec54d0f88266e667a3d3315'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:52.417841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b909cc15-6dfd-47d4-8d5e-093301c1382d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6c81addfec54d0f88266e667a3d3315'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:52.434060 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1100/2192] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:34:52 2022] DELETE /identity/v3/users/f6c81addfec54d0f88266e667a3d3315 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:52.955476 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cea022-4c5a-4c6e-8a3b-9d432ffd78dc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:52.957705 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a1cea022-4c5a-4c6e-8a3b-9d432ffd78dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:52.974863 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cea022-4c5a-4c6e-8a3b-9d432ffd78dc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:53.048127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86ba74-29bb-4d7a-90ac-9a42777f84b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:53.049611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86ba74-29bb-4d7a-90ac-9a42777f84b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:53.050085 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86ba74-29bb-4d7a-90ac-9a42777f84b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:53.050372 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86ba74-29bb-4d7a-90ac-9a42777f84b7 admin admin] PATH_INFO: `/v3/projects/0f0f92e91f6a432185c15b60980bd11e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:53.051747 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd86ba74-29bb-4d7a-90ac-9a42777f84b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f0f92e91f6a432185c15b60980bd11e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:53.053670 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd86ba74-29bb-4d7a-90ac-9a42777f84b7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:53.116412 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-dd86ba74-29bb-4d7a-90ac-9a42777f84b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f0f92e91f6a432185c15b60980bd11e', 'request_id': 'req-dd86ba74-29bb-4d7a-90ac-9a42777f84b7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:53.123770 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-dd86ba74-29bb-4d7a-90ac-9a42777f84b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f0f92e91f6a432185c15b60980bd11e', 'request_id': 'req-dd86ba74-29bb-4d7a-90ac-9a42777f84b7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:53.131119 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-dd86ba74-29bb-4d7a-90ac-9a42777f84b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f0f92e91f6a432185c15b60980bd11e', 'request_id': 'req-dd86ba74-29bb-4d7a-90ac-9a42777f84b7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:53.144524 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-dd86ba74-29bb-4d7a-90ac-9a42777f84b7 admin admin] The token cache is being invalidate because project 0f0f92e91f6a432185c15b60980bd11e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:53.144857 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-dd86ba74-29bb-4d7a-90ac-9a42777f84b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:53.145862 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-dd86ba74-29bb-4d7a-90ac-9a42777f84b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:53.147477 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1093/2193] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:52 2022] DELETE /identity/v3/projects/0f0f92e91f6a432185c15b60980bd11e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:53.152933 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad105de7-5672-4ffa-ad8c-bfff4a62b588 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:53.153178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad105de7-5672-4ffa-ad8c-bfff4a62b588 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:53.153460 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad105de7-5672-4ffa-ad8c-bfff4a62b588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:53.175017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ad105de7-5672-4ffa-ad8c-bfff4a62b588 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:53.176228 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ad105de7-5672-4ffa-ad8c-bfff4a62b588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:53.235912 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1101/2194] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:34:53.246253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63e231-6b27-43e2-9718-c10fe3b82953 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:53.246586 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63e231-6b27-43e2-9718-c10fe3b82953 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:53.246873 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63e231-6b27-43e2-9718-c10fe3b82953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:53.265655 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9b63e231-6b27-43e2-9718-c10fe3b82953 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:53.266885 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9b63e231-6b27-43e2-9718-c10fe3b82953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:53.275341 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1094/2195] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:53.279597 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad105de7-5672-4ffa-ad8c-bfff4a62b588 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:53.289871 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad105de7-5672-4ffa-ad8c-bfff4a62b588 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:53.294542 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da427b8-013c-45f7-ad73-ba5ca4ed9fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:53.295197 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da427b8-013c-45f7-ad73-ba5ca4ed9fa4 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:53.295488 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da427b8-013c-45f7-ad73-ba5ca4ed9fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:53.295768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da427b8-013c-45f7-ad73-ba5ca4ed9fa4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:53.296504 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da427b8-013c-45f7-ad73-ba5ca4ed9fa4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:34:53.296819 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da427b8-013c-45f7-ad73-ba5ca4ed9fa4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:53.298481 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da427b8-013c-45f7-ad73-ba5ca4ed9fa4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:53.310414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1102/2196] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:34:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:34:53.315138 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63e231-6b27-43e2-9718-c10fe3b82953 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:53.324447 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63e231-6b27-43e2-9718-c10fe3b82953 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:53.328993 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146210e6-579a-4a25-8007-9fc33ad20d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:53.329652 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146210e6-579a-4a25-8007-9fc33ad20d52 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:53.329971 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146210e6-579a-4a25-8007-9fc33ad20d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:53.330340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146210e6-579a-4a25-8007-9fc33ad20d52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:53.331075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146210e6-579a-4a25-8007-9fc33ad20d52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:53.333005 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146210e6-579a-4a25-8007-9fc33ad20d52 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:53.361597 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1095/2197] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:34:53 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:53.364874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da427b8-013c-45f7-ad73-ba5ca4ed9fa4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:53.375365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da427b8-013c-45f7-ad73-ba5ca4ed9fa4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:53.407662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:53.441430 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b9eda-88dd-42db-b9ed-dfa36b2ea070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:53.442107 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b9eda-88dd-42db-b9ed-dfa36b2ea070 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:53.442393 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b9eda-88dd-42db-b9ed-dfa36b2ea070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:53.442705 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b9eda-88dd-42db-b9ed-dfa36b2ea070 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:53.443462 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b9eda-88dd-42db-b9ed-dfa36b2ea070 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:53.445403 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b9eda-88dd-42db-b9ed-dfa36b2ea070 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:53.465881 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1103/2198] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:34:53 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:53.471163 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146210e6-579a-4a25-8007-9fc33ad20d52 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:53.481420 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146210e6-579a-4a25-8007-9fc33ad20d52 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:53.486177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19fba51-0ff6-4a40-b815-83a2d7e735dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:53.486934 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fba51-0ff6-4a40-b815-83a2d7e735dd admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:53.487220 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fba51-0ff6-4a40-b815-83a2d7e735dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:53.487502 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19fba51-0ff6-4a40-b815-83a2d7e735dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:53.488223 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19fba51-0ff6-4a40-b815-83a2d7e735dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:53.489824 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19fba51-0ff6-4a40-b815-83a2d7e735dd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:53.500199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1096/2199] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:53.504304 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b9eda-88dd-42db-b9ed-dfa36b2ea070 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:53.514098 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b9eda-88dd-42db-b9ed-dfa36b2ea070 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:53.519480 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db50aa-9d85-41f2-ac2d-15b3936b2ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:53.520142 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db50aa-9d85-41f2-ac2d-15b3936b2ec5 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:53.520703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db50aa-9d85-41f2-ac2d-15b3936b2ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:53.520963 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db50aa-9d85-41f2-ac2d-15b3936b2ec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:53.521700 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db50aa-9d85-41f2-ac2d-15b3936b2ec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:53.523344 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db50aa-9d85-41f2-ac2d-15b3936b2ec5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:53.533657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1104/2200] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:34:53.537802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19fba51-0ff6-4a40-b815-83a2d7e735dd admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:53.547669 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19fba51-0ff6-4a40-b815-83a2d7e735dd admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:53.552396 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107030ce-2310-4a2b-aaa0-1dd2a7d6f050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:53.553242 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107030ce-2310-4a2b-aaa0-1dd2a7d6f050 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:53.553525 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107030ce-2310-4a2b-aaa0-1dd2a7d6f050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:53.553805 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107030ce-2310-4a2b-aaa0-1dd2a7d6f050 admin admin] PATH_INFO: `/v3/projects/8836a392766a4528b65f28509501a33d/users/60857e6cb69e49b3b82e6f9186e80e5c/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:53.568799 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107030ce-2310-4a2b-aaa0-1dd2a7d6f050 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8836a392766a4528b65f28509501a33d, user_id=60857e6cb69e49b3b82e6f9186e80e5c, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:53.571510 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107030ce-2310-4a2b-aaa0-1dd2a7d6f050 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:53.584665 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1097/2201] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:53 2022] PUT /identity/v3/projects/8836a392766a4528b65f28509501a33d/users/60857e6cb69e49b3b82e6f9186e80e5c/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:53.594909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e42f3-61e2-4081-aa14-e286b7a5fca7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:53.595237 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e42f3-61e2-4081-aa14-e286b7a5fca7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:53.596126 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e42f3-61e2-4081-aa14-e286b7a5fca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:53.621886 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d88e42f3-61e2-4081-aa14-e286b7a5fca7 None None] MFA Rules not processed for user `60857e6cb69e49b3b82e6f9186e80e5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:53.623136 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d88e42f3-61e2-4081-aa14-e286b7a5fca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:53.687831 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1105/2202] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:53.692360 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107030ce-2310-4a2b-aaa0-1dd2a7d6f050 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:53.701159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:53.758252 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107030ce-2310-4a2b-aaa0-1dd2a7d6f050 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:53.762874 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d6c45-30c3-43a3-b8e7-397ee590ba05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:53.763529 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d6c45-30c3-43a3-b8e7-397ee590ba05 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:53.763850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d6c45-30c3-43a3-b8e7-397ee590ba05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:53.764139 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d6c45-30c3-43a3-b8e7-397ee590ba05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:53.764827 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1d6c45-30c3-43a3-b8e7-397ee590ba05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:53.766868 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1d6c45-30c3-43a3-b8e7-397ee590ba05 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:53.802586 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1098/2203] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:34:53 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:53.806407 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e42f3-61e2-4081-aa14-e286b7a5fca7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:53.821184 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e42f3-61e2-4081-aa14-e286b7a5fca7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:53.906335 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a4911-5059-402a-9d93-76c6063ce41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:53.907065 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a4911-5059-402a-9d93-76c6063ce41f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:53.907355 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a4911-5059-402a-9d93-76c6063ce41f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:53.907636 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5a4911-5059-402a-9d93-76c6063ce41f admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:53.908767 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a4911-5059-402a-9d93-76c6063ce41f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:53.911354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5a4911-5059-402a-9d93-76c6063ce41f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:53.940812 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1106/2204] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:34:53 2022] POST /identity/v3/users => generated 298 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:53.942673 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d6c45-30c3-43a3-b8e7-397ee590ba05 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:53.964360 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d6c45-30c3-43a3-b8e7-397ee590ba05 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:53.969522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca474e-babd-4847-8574-75e75ad95fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:53.970216 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca474e-babd-4847-8574-75e75ad95fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:53.971246 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca474e-babd-4847-8574-75e75ad95fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:53.971606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca474e-babd-4847-8574-75e75ad95fc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:53.972349 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca474e-babd-4847-8574-75e75ad95fc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:53.974076 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca474e-babd-4847-8574-75e75ad95fc0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:53.992594 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a4911-5059-402a-9d93-76c6063ce41f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:53.998072 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1099/2205] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:53 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:54.014006 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5a4911-5059-402a-9d93-76c6063ce41f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:54.019382 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc623a4-7136-4c6c-b888-0b2587be4532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:54.020665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc623a4-7136-4c6c-b888-0b2587be4532 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:54.021012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc623a4-7136-4c6c-b888-0b2587be4532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:54.021355 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc623a4-7136-4c6c-b888-0b2587be4532 admin admin] PATH_INFO: `/v3/projects/33b595067c5f4b46b7c073cfa1b86280/users/f8409ae03d134bc79731c46fde6dc5b4/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:54.042241 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc623a4-7136-4c6c-b888-0b2587be4532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33b595067c5f4b46b7c073cfa1b86280, user_id=f8409ae03d134bc79731c46fde6dc5b4, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:54.045020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc623a4-7136-4c6c-b888-0b2587be4532 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:54.062234 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1107/2206] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:53 2022] PUT /identity/v3/projects/33b595067c5f4b46b7c073cfa1b86280/users/f8409ae03d134bc79731c46fde6dc5b4/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:54.066326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca474e-babd-4847-8574-75e75ad95fc0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:54.077370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca474e-babd-4847-8574-75e75ad95fc0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:54.145126 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1d83b-feb3-4e76-912e-945f4d4caeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:54.146074 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1d83b-feb3-4e76-912e-945f4d4caeaf admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:54.146367 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1d83b-feb3-4e76-912e-945f4d4caeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:54.146692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1d83b-feb3-4e76-912e-945f4d4caeaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:54.147427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e1d83b-feb3-4e76-912e-945f4d4caeaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:54.149302 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e1d83b-feb3-4e76-912e-945f4d4caeaf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:54.163854 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1100/2207] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:54.167793 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc623a4-7136-4c6c-b888-0b2587be4532 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:54.181257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc623a4-7136-4c6c-b888-0b2587be4532 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:54.186243 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63dcc83-bee3-4249-9b73-e3692ed7233f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:54.187644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dcc83-bee3-4249-9b73-e3692ed7233f admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:54.187938 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dcc83-bee3-4249-9b73-e3692ed7233f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:54.188225 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63dcc83-bee3-4249-9b73-e3692ed7233f admin admin] PATH_INFO: `/v3/projects/33b595067c5f4b46b7c073cfa1b86280/users/f8409ae03d134bc79731c46fde6dc5b4/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:54.191009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63dcc83-bee3-4249-9b73-e3692ed7233f admin admin] RBAC: Authorizing `identity:create_grant(project_id=33b595067c5f4b46b7c073cfa1b86280, user_id=f8409ae03d134bc79731c46fde6dc5b4, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:54.193634 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63dcc83-bee3-4249-9b73-e3692ed7233f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:54.197096 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:54.209034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1108/2208] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:54 2022] PUT /identity/v3/projects/33b595067c5f4b46b7c073cfa1b86280/users/f8409ae03d134bc79731c46fde6dc5b4/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:54.220143 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868378c6-b20f-410f-b06d-69aee01c3c16 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:54.221052 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868378c6-b20f-410f-b06d-69aee01c3c16 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:54.221337 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868378c6-b20f-410f-b06d-69aee01c3c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:54.249028 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-868378c6-b20f-410f-b06d-69aee01c3c16 None None] MFA Rules not processed for user `f8409ae03d134bc79731c46fde6dc5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:54.250271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-868378c6-b20f-410f-b06d-69aee01c3c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:54.316277 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1101/2209] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:54.326809 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63dcc83-bee3-4249-9b73-e3692ed7233f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:54.328559 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a63dcc83-bee3-4249-9b73-e3692ed7233f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:54.386582 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63dcc83-bee3-4249-9b73-e3692ed7233f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:54.390191 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c09efe3-cf7f-4818-b0be-c4cc17a7cabb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:54.390870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09efe3-cf7f-4818-b0be-c4cc17a7cabb service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:54.391155 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09efe3-cf7f-4818-b0be-c4cc17a7cabb service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:54.391431 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c09efe3-cf7f-4818-b0be-c4cc17a7cabb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:54.401380 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c09efe3-cf7f-4818-b0be-c4cc17a7cabb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:54.402850 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c09efe3-cf7f-4818-b0be-c4cc17a7cabb service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:54.412716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1109/2210] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:54 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:54.929130 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868378c6-b20f-410f-b06d-69aee01c3c16 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:54.933266 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:54.994117 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868378c6-b20f-410f-b06d-69aee01c3c16 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:54.999053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b6cae-a9ed-4cf5-954d-1a5646d50580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:54.999935 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b6cae-a9ed-4cf5-954d-1a5646d50580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:55.000217 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b6cae-a9ed-4cf5-954d-1a5646d50580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:55.000492 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b6cae-a9ed-4cf5-954d-1a5646d50580 admin admin] PATH_INFO: `/v3/users/60857e6cb69e49b3b82e6f9186e80e5c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:55.002307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073b6cae-a9ed-4cf5-954d-1a5646d50580 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60857e6cb69e49b3b82e6f9186e80e5c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:55.004281 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073b6cae-a9ed-4cf5-954d-1a5646d50580 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:55.050503 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-073b6cae-a9ed-4cf5-954d-1a5646d50580 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60857e6cb69e49b3b82e6f9186e80e5c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:55.058039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-073b6cae-a9ed-4cf5-954d-1a5646d50580 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60857e6cb69e49b3b82e6f9186e80e5c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:55.070760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-073b6cae-a9ed-4cf5-954d-1a5646d50580 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60857e6cb69e49b3b82e6f9186e80e5c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:55.071405 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-073b6cae-a9ed-4cf5-954d-1a5646d50580 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60857e6cb69e49b3b82e6f9186e80e5c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:55.072012 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-073b6cae-a9ed-4cf5-954d-1a5646d50580 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60857e6cb69e49b3b82e6f9186e80e5c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:55.084343 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1102/2211] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:34:54 2022] DELETE /identity/v3/users/60857e6cb69e49b3b82e6f9186e80e5c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:55.088986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996de7e-4f69-49b7-9b98-45af6cb0cb25 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:55.089284 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996de7e-4f69-49b7-9b98-45af6cb0cb25 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:55.089570 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996de7e-4f69-49b7-9b98-45af6cb0cb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:55.107146 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b996de7e-4f69-49b7-9b98-45af6cb0cb25 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:55.108349 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b996de7e-4f69-49b7-9b98-45af6cb0cb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:55.171259 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1110/2212] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:55.195124 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b6cae-a9ed-4cf5-954d-1a5646d50580 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:55.196855 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-073b6cae-a9ed-4cf5-954d-1a5646d50580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:55.268909 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b6cae-a9ed-4cf5-954d-1a5646d50580 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:55.273241 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7ee51-bf58-49b2-a538-277c90139900 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:55.273889 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7ee51-bf58-49b2-a538-277c90139900 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:55.274171 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7ee51-bf58-49b2-a538-277c90139900 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:55.274465 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7ee51-bf58-49b2-a538-277c90139900 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:55.284507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7ee51-bf58-49b2-a538-277c90139900 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:55.285943 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7ee51-bf58-49b2-a538-277c90139900 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:55.295530 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1103/2213] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:55.566523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996de7e-4f69-49b7-9b98-45af6cb0cb25 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:55.567968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b996de7e-4f69-49b7-9b98-45af6cb0cb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:55.583939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996de7e-4f69-49b7-9b98-45af6cb0cb25 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:55.589708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0203dfa-1976-4709-b0b2-c56979dce60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:55.590630 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0203dfa-1976-4709-b0b2-c56979dce60d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:55.590963 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0203dfa-1976-4709-b0b2-c56979dce60d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:55.591152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0203dfa-1976-4709-b0b2-c56979dce60d admin admin] PATH_INFO: `/v3/projects/8836a392766a4528b65f28509501a33d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:55.592357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0203dfa-1976-4709-b0b2-c56979dce60d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8836a392766a4528b65f28509501a33d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:55.594258 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0203dfa-1976-4709-b0b2-c56979dce60d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:55.641671 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b0203dfa-1976-4709-b0b2-c56979dce60d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8836a392766a4528b65f28509501a33d', 'request_id': 'req-b0203dfa-1976-4709-b0b2-c56979dce60d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:55.649804 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b0203dfa-1976-4709-b0b2-c56979dce60d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8836a392766a4528b65f28509501a33d', 'request_id': 'req-b0203dfa-1976-4709-b0b2-c56979dce60d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:55.655984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b0203dfa-1976-4709-b0b2-c56979dce60d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8836a392766a4528b65f28509501a33d', 'request_id': 'req-b0203dfa-1976-4709-b0b2-c56979dce60d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:55.666545 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b0203dfa-1976-4709-b0b2-c56979dce60d admin admin] The token cache is being invalidate because project 8836a392766a4528b65f28509501a33d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:55.666834 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b0203dfa-1976-4709-b0b2-c56979dce60d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:55.667456 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b0203dfa-1976-4709-b0b2-c56979dce60d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:55.668964 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1111/2214] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:55 2022] DELETE /identity/v3/projects/8836a392766a4528b65f28509501a33d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:55.673120 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7ee51-bf58-49b2-a538-277c90139900 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:55.683533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7ee51-bf58-49b2-a538-277c90139900 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:55.702951 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:55.742319 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72375308-2c77-441d-9e88-06864c2a0aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:55.743161 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72375308-2c77-441d-9e88-06864c2a0aa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:55.743444 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72375308-2c77-441d-9e88-06864c2a0aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:55.743725 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72375308-2c77-441d-9e88-06864c2a0aa1 admin admin] PATH_INFO: `/v3/users/f8409ae03d134bc79731c46fde6dc5b4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:55.744917 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72375308-2c77-441d-9e88-06864c2a0aa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8409ae03d134bc79731c46fde6dc5b4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:55.746877 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72375308-2c77-441d-9e88-06864c2a0aa1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:55.792101 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-72375308-2c77-441d-9e88-06864c2a0aa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8409ae03d134bc79731c46fde6dc5b4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:55.799945 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-72375308-2c77-441d-9e88-06864c2a0aa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8409ae03d134bc79731c46fde6dc5b4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:55.811902 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-72375308-2c77-441d-9e88-06864c2a0aa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8409ae03d134bc79731c46fde6dc5b4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:55.812536 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-72375308-2c77-441d-9e88-06864c2a0aa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8409ae03d134bc79731c46fde6dc5b4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:55.813180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-72375308-2c77-441d-9e88-06864c2a0aa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8409ae03d134bc79731c46fde6dc5b4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:55.826007 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1104/2215] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:34:55 2022] DELETE /identity/v3/users/f8409ae03d134bc79731c46fde6dc5b4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:56.099959 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0203dfa-1976-4709-b0b2-c56979dce60d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:56.101447 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b0203dfa-1976-4709-b0b2-c56979dce60d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:56.115552 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0203dfa-1976-4709-b0b2-c56979dce60d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:56.137389 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:56.184456 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ce161-5a94-43b1-8c21-ea31eb956a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:56.185336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ce161-5a94-43b1-8c21-ea31eb956a55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:56.185737 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ce161-5a94-43b1-8c21-ea31eb956a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:56.186022 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ce161-5a94-43b1-8c21-ea31eb956a55 admin admin] PATH_INFO: `/v3/projects/33b595067c5f4b46b7c073cfa1b86280` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:56.187266 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ce161-5a94-43b1-8c21-ea31eb956a55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33b595067c5f4b46b7c073cfa1b86280)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:56.189334 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ce161-5a94-43b1-8c21-ea31eb956a55 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:56.242680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a37ce161-5a94-43b1-8c21-ea31eb956a55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33b595067c5f4b46b7c073cfa1b86280', 'request_id': 'req-a37ce161-5a94-43b1-8c21-ea31eb956a55'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:56.252842 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a37ce161-5a94-43b1-8c21-ea31eb956a55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33b595067c5f4b46b7c073cfa1b86280', 'request_id': 'req-a37ce161-5a94-43b1-8c21-ea31eb956a55'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:56.260871 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a37ce161-5a94-43b1-8c21-ea31eb956a55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33b595067c5f4b46b7c073cfa1b86280', 'request_id': 'req-a37ce161-5a94-43b1-8c21-ea31eb956a55'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:56.276788 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a37ce161-5a94-43b1-8c21-ea31eb956a55 admin admin] The token cache is being invalidate because project 33b595067c5f4b46b7c073cfa1b86280 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:56.276989 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a37ce161-5a94-43b1-8c21-ea31eb956a55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:56.277753 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a37ce161-5a94-43b1-8c21-ea31eb956a55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:56.279965 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1112/2216] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:56 2022] DELETE /identity/v3/projects/33b595067c5f4b46b7c073cfa1b86280 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:56.285922 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa75e5b-1740-48ff-9773-d01959616b40 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:56.286385 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa75e5b-1740-48ff-9773-d01959616b40 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:56.286795 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa75e5b-1740-48ff-9773-d01959616b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:56.309517 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-baa75e5b-1740-48ff-9773-d01959616b40 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:56.311304 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-baa75e5b-1740-48ff-9773-d01959616b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:56.425131 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1105/2217] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:56.427456 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bd2ac-d52a-4f22-be0a-46323db6b18f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:56.427873 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bd2ac-d52a-4f22-be0a-46323db6b18f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:56.428281 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bd2ac-d52a-4f22-be0a-46323db6b18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:56.448583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c68bd2ac-d52a-4f22-be0a-46323db6b18f None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:56.450224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c68bd2ac-d52a-4f22-be0a-46323db6b18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:56.459571 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1113/2218] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:56.464518 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa75e5b-1740-48ff-9773-d01959616b40 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:56.476533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa75e5b-1740-48ff-9773-d01959616b40 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:56.482082 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189df9e-327e-448e-8a8b-ced1b5888965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:56.482822 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189df9e-327e-448e-8a8b-ced1b5888965 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:56.483106 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189df9e-327e-448e-8a8b-ced1b5888965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:56.483646 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189df9e-327e-448e-8a8b-ced1b5888965 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:56.484317 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8189df9e-327e-448e-8a8b-ced1b5888965 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:34:56.484612 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8189df9e-327e-448e-8a8b-ced1b5888965 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:56.486545 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8189df9e-327e-448e-8a8b-ced1b5888965 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:56.502661 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1106/2219] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:34:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:34:56.509820 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bd2ac-d52a-4f22-be0a-46323db6b18f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:56.525246 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bd2ac-d52a-4f22-be0a-46323db6b18f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:56.532111 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c417bb3-03e5-4e20-9613-a62191e040f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:56.533000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c417bb3-03e5-4e20-9613-a62191e040f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:56.533293 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c417bb3-03e5-4e20-9613-a62191e040f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:56.533577 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c417bb3-03e5-4e20-9613-a62191e040f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:56.534489 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c417bb3-03e5-4e20-9613-a62191e040f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:56.536469 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c417bb3-03e5-4e20-9613-a62191e040f0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:56.586459 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1114/2220] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:34:56 2022] POST /identity/v3/projects => generated 385 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:56.590875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189df9e-327e-448e-8a8b-ced1b5888965 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:56.605949 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189df9e-327e-448e-8a8b-ced1b5888965 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:56.649498 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:56.695370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48703f-c2a9-4da9-848f-3976115e4927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:56.696060 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48703f-c2a9-4da9-848f-3976115e4927 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:56.696345 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48703f-c2a9-4da9-848f-3976115e4927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:56.696628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48703f-c2a9-4da9-848f-3976115e4927 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:56.697331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48703f-c2a9-4da9-848f-3976115e4927 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:56.699346 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48703f-c2a9-4da9-848f-3976115e4927 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:56.723687 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1107/2221] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:34:56 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:34:56.727812 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c417bb3-03e5-4e20-9613-a62191e040f0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:56.739686 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c417bb3-03e5-4e20-9613-a62191e040f0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:56.745971 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0fc531-3f0d-408b-b8ca-30e5b50bbc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:56.747041 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fc531-3f0d-408b-b8ca-30e5b50bbc8c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:56.747400 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fc531-3f0d-408b-b8ca-30e5b50bbc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:56.747909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fc531-3f0d-408b-b8ca-30e5b50bbc8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:56.749374 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0fc531-3f0d-408b-b8ca-30e5b50bbc8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:56.752168 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0fc531-3f0d-408b-b8ca-30e5b50bbc8c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:56.768405 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1115/2222] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:56.769833 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48703f-c2a9-4da9-848f-3976115e4927 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:56.781178 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48703f-c2a9-4da9-848f-3976115e4927 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:56.787784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e7bff4-b8f8-4b08-b836-37fd88704a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:56.788727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7bff4-b8f8-4b08-b836-37fd88704a90 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:56.789168 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7bff4-b8f8-4b08-b836-37fd88704a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:56.789547 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7bff4-b8f8-4b08-b836-37fd88704a90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:56.790462 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e7bff4-b8f8-4b08-b836-37fd88704a90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:56.792178 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e7bff4-b8f8-4b08-b836-37fd88704a90 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:56.816462 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1108/2223] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:56.820900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0fc531-3f0d-408b-b8ca-30e5b50bbc8c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:56.834276 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0fc531-3f0d-408b-b8ca-30e5b50bbc8c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:56.839620 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33a81b-7f5b-4831-8122-286efda55135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:56.840516 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33a81b-7f5b-4831-8122-286efda55135 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:56.840826 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33a81b-7f5b-4831-8122-286efda55135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:56.841106 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33a81b-7f5b-4831-8122-286efda55135 admin admin] PATH_INFO: `/v3/projects/8db701e62b12468db6cecf0acd29ee4d/users/4137c9869d3d4a5988899dbd98fd2000/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:56.860390 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a33a81b-7f5b-4831-8122-286efda55135 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8db701e62b12468db6cecf0acd29ee4d, user_id=4137c9869d3d4a5988899dbd98fd2000, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:56.864855 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a33a81b-7f5b-4831-8122-286efda55135 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:56.883087 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1116/2224] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:56 2022] PUT /identity/v3/projects/8db701e62b12468db6cecf0acd29ee4d/users/4137c9869d3d4a5988899dbd98fd2000/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:56.896744 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764d099-cbf4-4042-9cc7-4a897b306164 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:56.897036 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764d099-cbf4-4042-9cc7-4a897b306164 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:56.897326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e764d099-cbf4-4042-9cc7-4a897b306164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:56.923807 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e764d099-cbf4-4042-9cc7-4a897b306164 None None] MFA Rules not processed for user `4137c9869d3d4a5988899dbd98fd2000`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:56.925093 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e764d099-cbf4-4042-9cc7-4a897b306164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:56.994114 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1109/2225] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:56.998612 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33a81b-7f5b-4831-8122-286efda55135 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:57.047885 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:57.063135 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33a81b-7f5b-4831-8122-286efda55135 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:57.069004 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8ceb1-bb96-4746-9c72-1628ff565a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:57.069663 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8ceb1-bb96-4746-9c72-1628ff565a08 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:57.069946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8ceb1-bb96-4746-9c72-1628ff565a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:57.070220 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a8ceb1-bb96-4746-9c72-1628ff565a08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:57.070946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8ceb1-bb96-4746-9c72-1628ff565a08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:57.072832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a8ceb1-bb96-4746-9c72-1628ff565a08 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:57.100298 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1117/2226] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:34:56 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:57.104153 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e764d099-cbf4-4042-9cc7-4a897b306164 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:57.115375 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e764d099-cbf4-4042-9cc7-4a897b306164 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:57.178633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada14a72-1217-4f16-8ef6-525aa21a1bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:57.179345 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada14a72-1217-4f16-8ef6-525aa21a1bcc admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:57.179620 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada14a72-1217-4f16-8ef6-525aa21a1bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:57.179887 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada14a72-1217-4f16-8ef6-525aa21a1bcc admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:57.180576 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada14a72-1217-4f16-8ef6-525aa21a1bcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:57.182462 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada14a72-1217-4f16-8ef6-525aa21a1bcc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:57.204329 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1110/2227] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:34:57 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:34:57.209511 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8ceb1-bb96-4746-9c72-1628ff565a08 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:57.219824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a8ceb1-bb96-4746-9c72-1628ff565a08 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:57.225152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96d478-6365-4217-8ced-f7aa050f09b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:57.225836 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96d478-6365-4217-8ced-f7aa050f09b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:57.226130 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96d478-6365-4217-8ced-f7aa050f09b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:57.226416 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96d478-6365-4217-8ced-f7aa050f09b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:57.227172 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c96d478-6365-4217-8ced-f7aa050f09b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:57.228785 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c96d478-6365-4217-8ced-f7aa050f09b8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:57.239425 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1118/2228] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:57.243443 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada14a72-1217-4f16-8ef6-525aa21a1bcc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:57.253710 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada14a72-1217-4f16-8ef6-525aa21a1bcc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:57.258233 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f122c-2a00-41ce-b94a-16eaf1bd728b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:57.259219 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f122c-2a00-41ce-b94a-16eaf1bd728b admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:57.259505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f122c-2a00-41ce-b94a-16eaf1bd728b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:57.259785 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f122c-2a00-41ce-b94a-16eaf1bd728b admin admin] PATH_INFO: `/v3/projects/b8317294c48e4cb09969d1a6518cbb4a/users/54cf8039d365438d84b6b4572bbd9568/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:57.277099 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f122c-2a00-41ce-b94a-16eaf1bd728b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8317294c48e4cb09969d1a6518cbb4a, user_id=54cf8039d365438d84b6b4572bbd9568, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:57.280065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f122c-2a00-41ce-b94a-16eaf1bd728b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:57.296594 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1111/2229] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:57 2022] PUT /identity/v3/projects/b8317294c48e4cb09969d1a6518cbb4a/users/54cf8039d365438d84b6b4572bbd9568/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:34:57.301044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96d478-6365-4217-8ced-f7aa050f09b8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:57.311325 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96d478-6365-4217-8ced-f7aa050f09b8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:57.371449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66004d8-9a5c-4ae6-bb0c-104766d7b2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:57.372130 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66004d8-9a5c-4ae6-bb0c-104766d7b2c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:57.372422 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66004d8-9a5c-4ae6-bb0c-104766d7b2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:57.372737 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66004d8-9a5c-4ae6-bb0c-104766d7b2c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:57.373490 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66004d8-9a5c-4ae6-bb0c-104766d7b2c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:57.375283 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66004d8-9a5c-4ae6-bb0c-104766d7b2c3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:57.385953 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1119/2230] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:34:57 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:34:57.390144 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f122c-2a00-41ce-b94a-16eaf1bd728b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:57.397968 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:57.402173 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f122c-2a00-41ce-b94a-16eaf1bd728b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:57.409033 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9be2c-4203-4372-967a-9d44ec63cbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:57.410024 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9be2c-4203-4372-967a-9d44ec63cbbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:57.410368 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9be2c-4203-4372-967a-9d44ec63cbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:57.410777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c9be2c-4203-4372-967a-9d44ec63cbbb admin admin] PATH_INFO: `/v3/projects/b8317294c48e4cb09969d1a6518cbb4a/users/54cf8039d365438d84b6b4572bbd9568/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:57.412939 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c9be2c-4203-4372-967a-9d44ec63cbbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8317294c48e4cb09969d1a6518cbb4a, user_id=54cf8039d365438d84b6b4572bbd9568, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:57.415620 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c9be2c-4203-4372-967a-9d44ec63cbbb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:57.426072 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1112/2231] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:34:57 2022] PUT /identity/v3/projects/b8317294c48e4cb09969d1a6518cbb4a/users/54cf8039d365438d84b6b4572bbd9568/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:57.437183 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a2b75-7bdd-4fd9-88c7-91ccdee106c0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:57.437480 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a2b75-7bdd-4fd9-88c7-91ccdee106c0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:57.437767 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a2b75-7bdd-4fd9-88c7-91ccdee106c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:57.464208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-271a2b75-7bdd-4fd9-88c7-91ccdee106c0 None None] MFA Rules not processed for user `54cf8039d365438d84b6b4572bbd9568`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:57.465420 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-271a2b75-7bdd-4fd9-88c7-91ccdee106c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:57.536661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1120/2232] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:57.546962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9be2c-4203-4372-967a-9d44ec63cbbb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:57.548663 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b6c9be2c-4203-4372-967a-9d44ec63cbbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:57.609645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c9be2c-4203-4372-967a-9d44ec63cbbb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:57.613948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd68ca9b-72d5-4e06-b21e-44cee8a72af5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:57.614543 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68ca9b-72d5-4e06-b21e-44cee8a72af5 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:57.614830 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68ca9b-72d5-4e06-b21e-44cee8a72af5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:57.615107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd68ca9b-72d5-4e06-b21e-44cee8a72af5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:57.676113 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd68ca9b-72d5-4e06-b21e-44cee8a72af5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:57.677546 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd68ca9b-72d5-4e06-b21e-44cee8a72af5 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:57.686393 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1113/2233] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:58.219642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a2b75-7bdd-4fd9-88c7-91ccdee106c0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:58.221754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-271a2b75-7bdd-4fd9-88c7-91ccdee106c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:58.269426 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:58.290719 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a2b75-7bdd-4fd9-88c7-91ccdee106c0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:58.295280 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605c325-2b4c-411e-952c-5447176d2615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:58.296110 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605c325-2b4c-411e-952c-5447176d2615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:58.296394 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605c325-2b4c-411e-952c-5447176d2615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:58.296669 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605c325-2b4c-411e-952c-5447176d2615 admin admin] PATH_INFO: `/v3/users/99d1bd1790c049e5869f61cfd7b51d25` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:58.297854 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1605c325-2b4c-411e-952c-5447176d2615 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99d1bd1790c049e5869f61cfd7b51d25)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:58.299995 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1605c325-2b4c-411e-952c-5447176d2615 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:58.347092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1605c325-2b4c-411e-952c-5447176d2615 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99d1bd1790c049e5869f61cfd7b51d25'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:58.347683 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1605c325-2b4c-411e-952c-5447176d2615 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99d1bd1790c049e5869f61cfd7b51d25'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:58.360444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1605c325-2b4c-411e-952c-5447176d2615 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99d1bd1790c049e5869f61cfd7b51d25'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:58.361746 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1605c325-2b4c-411e-952c-5447176d2615 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99d1bd1790c049e5869f61cfd7b51d25'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:58.368885 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1605c325-2b4c-411e-952c-5447176d2615 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99d1bd1790c049e5869f61cfd7b51d25'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:58.382353 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1121/2234] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:34:58 2022] DELETE /identity/v3/users/99d1bd1790c049e5869f61cfd7b51d25 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:58.387061 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332cf314-b568-4127-9ba6-3eb027667237 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:58.387538 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332cf314-b568-4127-9ba6-3eb027667237 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:58.387731 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332cf314-b568-4127-9ba6-3eb027667237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:58.405850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-332cf314-b568-4127-9ba6-3eb027667237 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:58.407064 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-332cf314-b568-4127-9ba6-3eb027667237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:58.469421 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1114/2235] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:58.479169 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605c325-2b4c-411e-952c-5447176d2615 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:58.480559 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1605c325-2b4c-411e-952c-5447176d2615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:58.539509 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605c325-2b4c-411e-952c-5447176d2615 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:58.543201 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee8541-38f4-4f40-90ac-a9d286344c19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:58.543843 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee8541-38f4-4f40-90ac-a9d286344c19 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:58.544126 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee8541-38f4-4f40-90ac-a9d286344c19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:58.544400 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee8541-38f4-4f40-90ac-a9d286344c19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:58.554517 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee8541-38f4-4f40-90ac-a9d286344c19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:58.555910 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee8541-38f4-4f40-90ac-a9d286344c19 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:58.565664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1122/2236] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:58.661943 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332cf314-b568-4127-9ba6-3eb027667237 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:58.663900 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-332cf314-b568-4127-9ba6-3eb027667237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:58.679445 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332cf314-b568-4127-9ba6-3eb027667237 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:58.684229 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59e2b8-961f-468f-889e-9715d7526804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:58.685345 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59e2b8-961f-468f-889e-9715d7526804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:58.685465 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59e2b8-961f-468f-889e-9715d7526804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:58.685657 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be59e2b8-961f-468f-889e-9715d7526804 admin admin] PATH_INFO: `/v3/users/83a783c80c1c401d872ef22e6b461938` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:58.687003 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be59e2b8-961f-468f-889e-9715d7526804 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83a783c80c1c401d872ef22e6b461938)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:58.688912 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be59e2b8-961f-468f-889e-9715d7526804 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:58.741030 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:58.747544 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-be59e2b8-961f-468f-889e-9715d7526804 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83a783c80c1c401d872ef22e6b461938'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:58.756729 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-be59e2b8-961f-468f-889e-9715d7526804 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83a783c80c1c401d872ef22e6b461938'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:58.771503 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-be59e2b8-961f-468f-889e-9715d7526804 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83a783c80c1c401d872ef22e6b461938'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:58.772627 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-be59e2b8-961f-468f-889e-9715d7526804 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83a783c80c1c401d872ef22e6b461938'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:58.773278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-be59e2b8-961f-468f-889e-9715d7526804 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83a783c80c1c401d872ef22e6b461938'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:58.788816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1115/2237] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:34:58 2022] DELETE /identity/v3/users/83a783c80c1c401d872ef22e6b461938 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:58.793258 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b1c21-c62b-4787-9a39-1d51a9396cc4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:58.793555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b1c21-c62b-4787-9a39-1d51a9396cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:58.793840 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b1c21-c62b-4787-9a39-1d51a9396cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:58.812776 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-296b1c21-c62b-4787-9a39-1d51a9396cc4 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:34:58.814695 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-296b1c21-c62b-4787-9a39-1d51a9396cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:58.892543 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1123/2238] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:34:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:34:58.902578 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59e2b8-961f-468f-889e-9715d7526804 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:58.904156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-be59e2b8-961f-468f-889e-9715d7526804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:58.968750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be59e2b8-961f-468f-889e-9715d7526804 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:58.973154 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c963421-5ed3-4e98-8e88-b1b5c7667191 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:58.973812 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c963421-5ed3-4e98-8e88-b1b5c7667191 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:58.974107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c963421-5ed3-4e98-8e88-b1b5c7667191 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:58.974393 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c963421-5ed3-4e98-8e88-b1b5c7667191 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:58.984608 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c963421-5ed3-4e98-8e88-b1b5c7667191 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:58.986157 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c963421-5ed3-4e98-8e88-b1b5c7667191 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:58.997380 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1116/2239] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:34:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:34:59.005982 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b1c21-c62b-4787-9a39-1d51a9396cc4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:59.007944 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-296b1c21-c62b-4787-9a39-1d51a9396cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:59.023791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b1c21-c62b-4787-9a39-1d51a9396cc4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:59.031368 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96894f7d-86b6-494f-8b2d-b5be42846e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:59.032436 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96894f7d-86b6-494f-8b2d-b5be42846e2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:59.032686 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96894f7d-86b6-494f-8b2d-b5be42846e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:59.033637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96894f7d-86b6-494f-8b2d-b5be42846e2b admin admin] PATH_INFO: `/v3/projects/aebcb8dc38ef49da8f70ec08df95e12b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:59.035014 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96894f7d-86b6-494f-8b2d-b5be42846e2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=aebcb8dc38ef49da8f70ec08df95e12b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:59.037012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96894f7d-86b6-494f-8b2d-b5be42846e2b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:59.050676 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:59.121558 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-96894f7d-86b6-494f-8b2d-b5be42846e2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aebcb8dc38ef49da8f70ec08df95e12b', 'request_id': 'req-96894f7d-86b6-494f-8b2d-b5be42846e2b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:59.132619 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-96894f7d-86b6-494f-8b2d-b5be42846e2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aebcb8dc38ef49da8f70ec08df95e12b', 'request_id': 'req-96894f7d-86b6-494f-8b2d-b5be42846e2b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:59.143606 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-96894f7d-86b6-494f-8b2d-b5be42846e2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aebcb8dc38ef49da8f70ec08df95e12b', 'request_id': 'req-96894f7d-86b6-494f-8b2d-b5be42846e2b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:59.157946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-96894f7d-86b6-494f-8b2d-b5be42846e2b admin admin] The token cache is being invalidate because project aebcb8dc38ef49da8f70ec08df95e12b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:59.158254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-96894f7d-86b6-494f-8b2d-b5be42846e2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:59.159516 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-96894f7d-86b6-494f-8b2d-b5be42846e2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:59.166568 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1124/2240] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:59 2022] DELETE /identity/v3/projects/aebcb8dc38ef49da8f70ec08df95e12b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:34:59.569119 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c963421-5ed3-4e98-8e88-b1b5c7667191 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:34:59.570634 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8c963421-5ed3-4e98-8e88-b1b5c7667191 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:34:59.604289 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c963421-5ed3-4e98-8e88-b1b5c7667191 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:34:59.683794 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583fe583-bc8a-4ef8-9de7-02f23c0dbc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:34:59.685424 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583fe583-bc8a-4ef8-9de7-02f23c0dbc7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:34:59.686257 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583fe583-bc8a-4ef8-9de7-02f23c0dbc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:34:59.686619 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583fe583-bc8a-4ef8-9de7-02f23c0dbc7a admin admin] PATH_INFO: `/v3/projects/b51e48337ed549918fec2c24595597f0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:34:59.690271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583fe583-bc8a-4ef8-9de7-02f23c0dbc7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b51e48337ed549918fec2c24595597f0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:34:59.692514 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583fe583-bc8a-4ef8-9de7-02f23c0dbc7a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:34:59.773703 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-583fe583-bc8a-4ef8-9de7-02f23c0dbc7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b51e48337ed549918fec2c24595597f0', 'request_id': 'req-583fe583-bc8a-4ef8-9de7-02f23c0dbc7a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:59.783879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-583fe583-bc8a-4ef8-9de7-02f23c0dbc7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b51e48337ed549918fec2c24595597f0', 'request_id': 'req-583fe583-bc8a-4ef8-9de7-02f23c0dbc7a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:59.790514 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-583fe583-bc8a-4ef8-9de7-02f23c0dbc7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b51e48337ed549918fec2c24595597f0', 'request_id': 'req-583fe583-bc8a-4ef8-9de7-02f23c0dbc7a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:59.792155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:34:59.813502 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-583fe583-bc8a-4ef8-9de7-02f23c0dbc7a admin admin] The token cache is being invalidate because project b51e48337ed549918fec2c24595597f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:34:59.813912 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-583fe583-bc8a-4ef8-9de7-02f23c0dbc7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:59.815387 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-583fe583-bc8a-4ef8-9de7-02f23c0dbc7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:34:59.818508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1117/2241] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:34:59 2022] DELETE /identity/v3/projects/b51e48337ed549918fec2c24595597f0 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:01.853185 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9731fa-4a8f-4ed9-8f8a-21147362a516 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:01.853682 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9731fa-4a8f-4ed9-8f8a-21147362a516 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:01.853770 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9731fa-4a8f-4ed9-8f8a-21147362a516 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:01.855242 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1125/2242] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jun 29 12:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:35:01.867186 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bcd20-2e20-4a36-bde8-b580a0cf7ed0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:01.867697 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bcd20-2e20-4a36-bde8-b580a0cf7ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:01.867909 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bcd20-2e20-4a36-bde8-b580a0cf7ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:01.889248 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-310bcd20-2e20-4a36-bde8-b580a0cf7ed0 None None] MFA Rules not processed for user `c98135cacb9d4f22846ad33500b9bd1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:01.890512 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-310bcd20-2e20-4a36-bde8-b580a0cf7ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:01.974705 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1118/2243] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jun 29 12:35:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:35:01.985933 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9731fa-4a8f-4ed9-8f8a-21147362a516 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:01.987983 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4f9731fa-4a8f-4ed9-8f8a-21147362a516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:02.005858 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9731fa-4a8f-4ed9-8f8a-21147362a516 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:02.011370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ddd73-f074-4ba1-957c-b653ba3c2c57 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98135cacb9d4f22846ad33500b9bd1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:02.012026 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ddd73-f074-4ba1-957c-b653ba3c2c57 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:02.012313 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ddd73-f074-4ba1-957c-b653ba3c2c57 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:02.012594 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ddd73-f074-4ba1-957c-b653ba3c2c57 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:02.022893 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ddd73-f074-4ba1-957c-b653ba3c2c57 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:02.024483 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ddd73-f074-4ba1-957c-b653ba3c2c57 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:02.035315 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1126/2244] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:35:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:35:02.303977 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bcd20-2e20-4a36-bde8-b580a0cf7ed0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:02.305366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-310bcd20-2e20-4a36-bde8-b580a0cf7ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:02.367032 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bcd20-2e20-4a36-bde8-b580a0cf7ed0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:02.370673 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c82c472-ab53-405b-a593-6aa981c07487 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:02.371334 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82c472-ab53-405b-a593-6aa981c07487 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:02.371614 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82c472-ab53-405b-a593-6aa981c07487 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:02.371895 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82c472-ab53-405b-a593-6aa981c07487 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:02.373758 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7c82c472-ab53-405b-a593-6aa981c07487 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:02.444044 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c82c472-ab53-405b-a593-6aa981c07487 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:02.445520 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c82c472-ab53-405b-a593-6aa981c07487 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:02.457484 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1119/2245] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:35:02 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:35:02.727677 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ddd73-f074-4ba1-957c-b653ba3c2c57 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:02.729272 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-371ddd73-f074-4ba1-957c-b653ba3c2c57 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:02.822580 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ddd73-f074-4ba1-957c-b653ba3c2c57 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:02.827135 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc723bc2-cb58-42b2-ae82-41be0e2b202b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:02.828008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc723bc2-cb58-42b2-ae82-41be0e2b202b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:02.828297 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc723bc2-cb58-42b2-ae82-41be0e2b202b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:02.828581 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc723bc2-cb58-42b2-ae82-41be0e2b202b admin admin] PATH_INFO: `/v3/users/fa9383b7671347e98bdc481c2d0c6486` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:02.830704 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc723bc2-cb58-42b2-ae82-41be0e2b202b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa9383b7671347e98bdc481c2d0c6486)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:02.832621 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc723bc2-cb58-42b2-ae82-41be0e2b202b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:02.886106 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fc723bc2-cb58-42b2-ae82-41be0e2b202b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa9383b7671347e98bdc481c2d0c6486'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:02.886798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fc723bc2-cb58-42b2-ae82-41be0e2b202b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa9383b7671347e98bdc481c2d0c6486'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:02.899308 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fc723bc2-cb58-42b2-ae82-41be0e2b202b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa9383b7671347e98bdc481c2d0c6486'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:02.899943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fc723bc2-cb58-42b2-ae82-41be0e2b202b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa9383b7671347e98bdc481c2d0c6486'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:02.906786 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fc723bc2-cb58-42b2-ae82-41be0e2b202b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa9383b7671347e98bdc481c2d0c6486'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:02.911570 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:02.922561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1127/2246] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:35:02 2022] DELETE /identity/v3/users/fa9383b7671347e98bdc481c2d0c6486 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:03.106805 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c82c472-ab53-405b-a593-6aa981c07487 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:03.118445 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c82c472-ab53-405b-a593-6aa981c07487 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:03.167710 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:03.195176 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70377ebc-1f51-430b-8ec0-5e9eb62894b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:03.196009 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70377ebc-1f51-430b-8ec0-5e9eb62894b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:03.196280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70377ebc-1f51-430b-8ec0-5e9eb62894b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:03.196566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70377ebc-1f51-430b-8ec0-5e9eb62894b7 admin admin] PATH_INFO: `/v3/projects/698adaaf9d124670be6ccdcb3cbd7ece` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:03.197775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70377ebc-1f51-430b-8ec0-5e9eb62894b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=698adaaf9d124670be6ccdcb3cbd7ece)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:03.199756 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70377ebc-1f51-430b-8ec0-5e9eb62894b7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:03.256147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-70377ebc-1f51-430b-8ec0-5e9eb62894b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '698adaaf9d124670be6ccdcb3cbd7ece', 'request_id': 'req-70377ebc-1f51-430b-8ec0-5e9eb62894b7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:03.262256 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-70377ebc-1f51-430b-8ec0-5e9eb62894b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '698adaaf9d124670be6ccdcb3cbd7ece', 'request_id': 'req-70377ebc-1f51-430b-8ec0-5e9eb62894b7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:03.268670 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-70377ebc-1f51-430b-8ec0-5e9eb62894b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '698adaaf9d124670be6ccdcb3cbd7ece', 'request_id': 'req-70377ebc-1f51-430b-8ec0-5e9eb62894b7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:03.285534 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-70377ebc-1f51-430b-8ec0-5e9eb62894b7 admin admin] The token cache is being invalidate because project 698adaaf9d124670be6ccdcb3cbd7ece was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:35:03.285827 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-70377ebc-1f51-430b-8ec0-5e9eb62894b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:03.286483 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-70377ebc-1f51-430b-8ec0-5e9eb62894b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:03.288045 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1120/2247] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:35:03 2022] DELETE /identity/v3/projects/698adaaf9d124670be6ccdcb3cbd7ece => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:03.292484 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc723bc2-cb58-42b2-ae82-41be0e2b202b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:03.293871 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fc723bc2-cb58-42b2-ae82-41be0e2b202b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:03.307069 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc723bc2-cb58-42b2-ae82-41be0e2b202b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:03.365373 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf8ca1e-ca3f-4514-a48b-0532b4f6057c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:03.366207 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8ca1e-ca3f-4514-a48b-0532b4f6057c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:03.366527 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8ca1e-ca3f-4514-a48b-0532b4f6057c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:03.366806 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf8ca1e-ca3f-4514-a48b-0532b4f6057c admin admin] PATH_INFO: `/v3/users/027c1ae7597640929980c680ecc071a4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:03.367986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf8ca1e-ca3f-4514-a48b-0532b4f6057c admin admin] RBAC: Authorizing `identity:delete_user(user_id=027c1ae7597640929980c680ecc071a4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:03.369907 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf8ca1e-ca3f-4514-a48b-0532b4f6057c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:03.425422 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bdf8ca1e-ca3f-4514-a48b-0532b4f6057c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '027c1ae7597640929980c680ecc071a4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:03.426077 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bdf8ca1e-ca3f-4514-a48b-0532b4f6057c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '027c1ae7597640929980c680ecc071a4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:03.437760 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bdf8ca1e-ca3f-4514-a48b-0532b4f6057c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '027c1ae7597640929980c680ecc071a4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:03.438386 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bdf8ca1e-ca3f-4514-a48b-0532b4f6057c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '027c1ae7597640929980c680ecc071a4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:03.445485 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bdf8ca1e-ca3f-4514-a48b-0532b4f6057c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '027c1ae7597640929980c680ecc071a4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:03.458711 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1128/2248] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:35:03 2022] DELETE /identity/v3/users/027c1ae7597640929980c680ecc071a4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:03.640042 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70377ebc-1f51-430b-8ec0-5e9eb62894b7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:03.641641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-70377ebc-1f51-430b-8ec0-5e9eb62894b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:03.660043 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70377ebc-1f51-430b-8ec0-5e9eb62894b7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:03.719613 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad09ca5-decc-4319-87a9-f46ab66fac38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:03.720915 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad09ca5-decc-4319-87a9-f46ab66fac38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:03.721203 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad09ca5-decc-4319-87a9-f46ab66fac38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:03.721483 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad09ca5-decc-4319-87a9-f46ab66fac38 admin admin] PATH_INFO: `/v3/projects/6026d9a909dc492cadfc3d899085940e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:03.722701 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad09ca5-decc-4319-87a9-f46ab66fac38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6026d9a909dc492cadfc3d899085940e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:03.724615 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad09ca5-decc-4319-87a9-f46ab66fac38 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:03.771202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ad09ca5-decc-4319-87a9-f46ab66fac38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6026d9a909dc492cadfc3d899085940e', 'request_id': 'req-5ad09ca5-decc-4319-87a9-f46ab66fac38'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:03.777666 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ad09ca5-decc-4319-87a9-f46ab66fac38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6026d9a909dc492cadfc3d899085940e', 'request_id': 'req-5ad09ca5-decc-4319-87a9-f46ab66fac38'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:03.783865 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ad09ca5-decc-4319-87a9-f46ab66fac38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6026d9a909dc492cadfc3d899085940e', 'request_id': 'req-5ad09ca5-decc-4319-87a9-f46ab66fac38'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:03.792657 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:03.799391 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ad09ca5-decc-4319-87a9-f46ab66fac38 admin admin] The token cache is being invalidate because project 6026d9a909dc492cadfc3d899085940e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:35:03.799877 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ad09ca5-decc-4319-87a9-f46ab66fac38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:03.801108 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ad09ca5-decc-4319-87a9-f46ab66fac38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:03.803602 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1121/2249] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:35:03 2022] DELETE /identity/v3/projects/6026d9a909dc492cadfc3d899085940e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:03.810019 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da67ec-4e48-45a6-96fc-fe8d87f6ab7a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:03.810321 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da67ec-4e48-45a6-96fc-fe8d87f6ab7a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:03.810662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da67ec-4e48-45a6-96fc-fe8d87f6ab7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:03.828297 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-72da67ec-4e48-45a6-96fc-fe8d87f6ab7a None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:03.829649 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-72da67ec-4e48-45a6-96fc-fe8d87f6ab7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:03.890693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1129/2250] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:35:03.957153 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29aeaf-fea0-4339-a859-826b2d69c069 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:03.957439 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29aeaf-fea0-4339-a859-826b2d69c069 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:03.957780 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea29aeaf-fea0-4339-a859-826b2d69c069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:03.981787 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ea29aeaf-fea0-4339-a859-826b2d69c069 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:03.982938 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ea29aeaf-fea0-4339-a859-826b2d69c069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:03.990880 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1122/2251] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:35:03.995338 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da67ec-4e48-45a6-96fc-fe8d87f6ab7a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:04.005375 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da67ec-4e48-45a6-96fc-fe8d87f6ab7a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:04.009965 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816606e-df1e-4c2f-921d-0d5f90a535b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:04.010663 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816606e-df1e-4c2f-921d-0d5f90a535b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:04.010955 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816606e-df1e-4c2f-921d-0d5f90a535b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:04.011236 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816606e-df1e-4c2f-921d-0d5f90a535b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:04.011972 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816606e-df1e-4c2f-921d-0d5f90a535b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:35:04.012288 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816606e-df1e-4c2f-921d-0d5f90a535b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:04.013900 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816606e-df1e-4c2f-921d-0d5f90a535b3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:04.026332 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1130/2252] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:35:04.030768 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29aeaf-fea0-4339-a859-826b2d69c069 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:04.041401 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea29aeaf-fea0-4339-a859-826b2d69c069 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:04.045974 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b8821-d2f2-4f95-a9fd-836c6ddb30ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:04.046651 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b8821-d2f2-4f95-a9fd-836c6ddb30ed admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:04.046939 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b8821-d2f2-4f95-a9fd-836c6ddb30ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:04.047221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b8821-d2f2-4f95-a9fd-836c6ddb30ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:04.047953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4b8821-d2f2-4f95-a9fd-836c6ddb30ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:04.050406 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4b8821-d2f2-4f95-a9fd-836c6ddb30ed admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:04.077455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1123/2253] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:35:04 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:35:04.082391 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816606e-df1e-4c2f-921d-0d5f90a535b3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:04.087553 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:04.093823 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816606e-df1e-4c2f-921d-0d5f90a535b3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:04.155794 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877561e8-5c54-4e6b-81b1-060d879cefab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:04.156589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877561e8-5c54-4e6b-81b1-060d879cefab admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:04.156768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877561e8-5c54-4e6b-81b1-060d879cefab admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:04.157050 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877561e8-5c54-4e6b-81b1-060d879cefab admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:04.159065 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877561e8-5c54-4e6b-81b1-060d879cefab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:04.160889 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877561e8-5c54-4e6b-81b1-060d879cefab admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:04.183390 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1131/2254] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:35:04 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:35:04.188163 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b8821-d2f2-4f95-a9fd-836c6ddb30ed admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:04.201714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b8821-d2f2-4f95-a9fd-836c6ddb30ed admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:04.208051 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19836c2c-4f4b-474e-b718-adb5a678c94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:04.208727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19836c2c-4f4b-474e-b718-adb5a678c94b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:04.209014 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19836c2c-4f4b-474e-b718-adb5a678c94b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:04.209294 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19836c2c-4f4b-474e-b718-adb5a678c94b admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:04.210024 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19836c2c-4f4b-474e-b718-adb5a678c94b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:04.211661 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19836c2c-4f4b-474e-b718-adb5a678c94b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:04.223247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1124/2255] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:35:04.228021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877561e8-5c54-4e6b-81b1-060d879cefab admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:04.239079 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877561e8-5c54-4e6b-81b1-060d879cefab admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:04.244962 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7bdde-d66e-4fc4-96a0-50b17025ab11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:04.245666 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7bdde-d66e-4fc4-96a0-50b17025ab11 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:04.245985 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7bdde-d66e-4fc4-96a0-50b17025ab11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:04.246274 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7bdde-d66e-4fc4-96a0-50b17025ab11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:04.247615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7bdde-d66e-4fc4-96a0-50b17025ab11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:04.248701 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7bdde-d66e-4fc4-96a0-50b17025ab11 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:04.261551 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1132/2256] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:35:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:35:04.265592 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19836c2c-4f4b-474e-b718-adb5a678c94b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:04.276906 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19836c2c-4f4b-474e-b718-adb5a678c94b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:04.281615 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17848da6-0981-47bc-baca-72e083d52cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:04.282693 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17848da6-0981-47bc-baca-72e083d52cd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:04.283274 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17848da6-0981-47bc-baca-72e083d52cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:04.283274 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17848da6-0981-47bc-baca-72e083d52cd0 admin admin] PATH_INFO: `/v3/projects/aacced43dbbb4c2e98bdff0d3cc557ad/users/f026018d7f444519983df75eb4453284/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:04.299314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17848da6-0981-47bc-baca-72e083d52cd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aacced43dbbb4c2e98bdff0d3cc557ad, user_id=f026018d7f444519983df75eb4453284, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:04.302001 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17848da6-0981-47bc-baca-72e083d52cd0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:04.316663 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1125/2257] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:35:04 2022] PUT /identity/v3/projects/aacced43dbbb4c2e98bdff0d3cc557ad/users/f026018d7f444519983df75eb4453284/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:04.322334 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855beecd-18f8-4934-addc-77ff4065d142 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:04.322734 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855beecd-18f8-4934-addc-77ff4065d142 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:04.323084 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855beecd-18f8-4934-addc-77ff4065d142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:04.343953 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-855beecd-18f8-4934-addc-77ff4065d142 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:04.345349 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-855beecd-18f8-4934-addc-77ff4065d142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:04.410311 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1133/2258] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:35:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:35:04.419889 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17848da6-0981-47bc-baca-72e083d52cd0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:04.421710 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-17848da6-0981-47bc-baca-72e083d52cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:04.480296 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17848da6-0981-47bc-baca-72e083d52cd0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:04.483978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4ee6d-74ef-40ca-bdd5-40ed12beb7f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:04.484628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4ee6d-74ef-40ca-bdd5-40ed12beb7f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:04.484916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4ee6d-74ef-40ca-bdd5-40ed12beb7f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:04.485195 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4ee6d-74ef-40ca-bdd5-40ed12beb7f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:04.494738 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee4ee6d-74ef-40ca-bdd5-40ed12beb7f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:04.496331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee4ee6d-74ef-40ca-bdd5-40ed12beb7f5 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:04.506587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1126/2259] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:35:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:35:05.230908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d7c84-9828-43fc-bc93-ae9c31088161 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:05.231534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d7c84-9828-43fc-bc93-ae9c31088161 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:05.231534 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d7c84-9828-43fc-bc93-ae9c31088161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:05.255258 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-aa6d7c84-9828-43fc-bc93-ae9c31088161 None None] MFA Rules not processed for user `8c10852a728844c6ae0a48e3772d95b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:05.256768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-aa6d7c84-9828-43fc-bc93-ae9c31088161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:05.351251 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1134/2260] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jun 29 12:35:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:35:05.420472 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4ee6d-74ef-40ca-bdd5-40ed12beb7f5 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:05.422019 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1ee4ee6d-74ef-40ca-bdd5-40ed12beb7f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:05.556892 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4ee6d-74ef-40ca-bdd5-40ed12beb7f5 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:05.564877 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d632c-1acf-401a-b8d5-d5fd2674175e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209b0df45f76404da880a9ae0c637d67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:05.565680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d632c-1acf-401a-b8d5-d5fd2674175e service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:05.566208 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d632c-1acf-401a-b8d5-d5fd2674175e service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:05.567170 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d632c-1acf-401a-b8d5-d5fd2674175e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:05.582147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d632c-1acf-401a-b8d5-d5fd2674175e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:05.585912 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d632c-1acf-401a-b8d5-d5fd2674175e service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:05.619257 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1127/2261] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 12:35:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 12:35:08.701355 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d7c84-9828-43fc-bc93-ae9c31088161 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:08.703056 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-aa6d7c84-9828-43fc-bc93-ae9c31088161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:08.732312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d7c84-9828-43fc-bc93-ae9c31088161 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:08.747566 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c10c5b-de9c-4803-948c-8cf4285f4866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:08.748587 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c10c5b-de9c-4803-948c-8cf4285f4866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:08.748775 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c10c5b-de9c-4803-948c-8cf4285f4866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:08.749037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c10c5b-de9c-4803-948c-8cf4285f4866 admin admin] PATH_INFO: `/v3/users/4137c9869d3d4a5988899dbd98fd2000` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:08.751656 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c10c5b-de9c-4803-948c-8cf4285f4866 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4137c9869d3d4a5988899dbd98fd2000)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:08.753710 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c10c5b-de9c-4803-948c-8cf4285f4866 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:08.803664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:08.851242 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-63c10c5b-de9c-4803-948c-8cf4285f4866 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4137c9869d3d4a5988899dbd98fd2000'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:08.854641 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-63c10c5b-de9c-4803-948c-8cf4285f4866 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4137c9869d3d4a5988899dbd98fd2000'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:08.877235 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-63c10c5b-de9c-4803-948c-8cf4285f4866 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4137c9869d3d4a5988899dbd98fd2000'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:08.889481 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-63c10c5b-de9c-4803-948c-8cf4285f4866 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4137c9869d3d4a5988899dbd98fd2000'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:08.898360 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-63c10c5b-de9c-4803-948c-8cf4285f4866 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4137c9869d3d4a5988899dbd98fd2000'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:08.925686 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1135/2262] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:35:08 2022] DELETE /identity/v3/users/4137c9869d3d4a5988899dbd98fd2000 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:08.927712 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6b502-74c6-4ca3-81cc-344aed300676 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:08.928236 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6b502-74c6-4ca3-81cc-344aed300676 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:08.928864 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6b502-74c6-4ca3-81cc-344aed300676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:08.971371 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-61a6b502-74c6-4ca3-81cc-344aed300676 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:08.975093 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-61a6b502-74c6-4ca3-81cc-344aed300676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:09.062071 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:09.165502 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1128/2263] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:35:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:35:09.175865 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c10c5b-de9c-4803-948c-8cf4285f4866 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:09.178376 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-63c10c5b-de9c-4803-948c-8cf4285f4866 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:09.323080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c10c5b-de9c-4803-948c-8cf4285f4866 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:09.336393 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109ba8f-015b-450c-bcaf-400e1923befc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:09.337077 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109ba8f-015b-450c-bcaf-400e1923befc service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:09.337421 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109ba8f-015b-450c-bcaf-400e1923befc service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:09.337703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109ba8f-015b-450c-bcaf-400e1923befc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:09.388943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c109ba8f-015b-450c-bcaf-400e1923befc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:09.394362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c109ba8f-015b-450c-bcaf-400e1923befc service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:09.422009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1136/2264] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:35:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 12:35:10.327856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6b502-74c6-4ca3-81cc-344aed300676 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:10.329892 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-61a6b502-74c6-4ca3-81cc-344aed300676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:10.356981 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6b502-74c6-4ca3-81cc-344aed300676 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:10.365711 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b62b6e3-8018-4156-9854-3efe77b5abde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:10.366800 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62b6e3-8018-4156-9854-3efe77b5abde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:10.367235 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62b6e3-8018-4156-9854-3efe77b5abde admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:10.367781 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b62b6e3-8018-4156-9854-3efe77b5abde admin admin] PATH_INFO: `/v3/projects/8db701e62b12468db6cecf0acd29ee4d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:10.370837 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b62b6e3-8018-4156-9854-3efe77b5abde admin admin] RBAC: Authorizing `identity:delete_project(project_id=8db701e62b12468db6cecf0acd29ee4d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:10.374522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b62b6e3-8018-4156-9854-3efe77b5abde admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:10.507992 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3b62b6e3-8018-4156-9854-3efe77b5abde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8db701e62b12468db6cecf0acd29ee4d', 'request_id': 'req-3b62b6e3-8018-4156-9854-3efe77b5abde'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:10.519174 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3b62b6e3-8018-4156-9854-3efe77b5abde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8db701e62b12468db6cecf0acd29ee4d', 'request_id': 'req-3b62b6e3-8018-4156-9854-3efe77b5abde'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:10.535324 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3b62b6e3-8018-4156-9854-3efe77b5abde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8db701e62b12468db6cecf0acd29ee4d', 'request_id': 'req-3b62b6e3-8018-4156-9854-3efe77b5abde'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:10.556622 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3b62b6e3-8018-4156-9854-3efe77b5abde admin admin] The token cache is being invalidate because project 8db701e62b12468db6cecf0acd29ee4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:35:10.556922 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3b62b6e3-8018-4156-9854-3efe77b5abde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:10.557643 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3b62b6e3-8018-4156-9854-3efe77b5abde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:10.564232 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1129/2265] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:35:10 2022] DELETE /identity/v3/projects/8db701e62b12468db6cecf0acd29ee4d => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:10.569028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109ba8f-015b-450c-bcaf-400e1923befc service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:10.582617 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109ba8f-015b-450c-bcaf-400e1923befc service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:10.686694 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca2f82-96b4-4516-b39c-a841df5739e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:10.687597 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca2f82-96b4-4516-b39c-a841df5739e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:10.687889 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca2f82-96b4-4516-b39c-a841df5739e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:10.688211 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca2f82-96b4-4516-b39c-a841df5739e8 admin admin] PATH_INFO: `/v3/users/54cf8039d365438d84b6b4572bbd9568` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:10.690083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca2f82-96b4-4516-b39c-a841df5739e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54cf8039d365438d84b6b4572bbd9568)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:10.693944 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cca2f82-96b4-4516-b39c-a841df5739e8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:10.783802 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:10.790334 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9cca2f82-96b4-4516-b39c-a841df5739e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54cf8039d365438d84b6b4572bbd9568'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:10.794306 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9cca2f82-96b4-4516-b39c-a841df5739e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54cf8039d365438d84b6b4572bbd9568'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:10.825451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99b42e-cebc-4d50-9e69-1e6b49be34ee None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:10.826086 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99b42e-cebc-4d50-9e69-1e6b49be34ee None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:10.826572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99b42e-cebc-4d50-9e69-1e6b49be34ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:10.831075 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9cca2f82-96b4-4516-b39c-a841df5739e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54cf8039d365438d84b6b4572bbd9568'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:10.832097 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9cca2f82-96b4-4516-b39c-a841df5739e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54cf8039d365438d84b6b4572bbd9568'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:10.843215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9cca2f82-96b4-4516-b39c-a841df5739e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54cf8039d365438d84b6b4572bbd9568'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:10.867246 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1137/2266] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:35:10 2022] DELETE /identity/v3/users/54cf8039d365438d84b6b4572bbd9568 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:10.900623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5c99b42e-cebc-4d50-9e69-1e6b49be34ee None None] MFA Rules not processed for user `f026018d7f444519983df75eb4453284`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:10.902635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5c99b42e-cebc-4d50-9e69-1e6b49be34ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:11.086120 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1130/2267] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:35:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:35:11.088788 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca2f82-96b4-4516-b39c-a841df5739e8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:11.091668 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9cca2f82-96b4-4516-b39c-a841df5739e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:11.221478 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca2f82-96b4-4516-b39c-a841df5739e8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:11.228580 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd5267-aaa0-4c0b-95dc-f9c659311cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:11.229718 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd5267-aaa0-4c0b-95dc-f9c659311cfb admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:11.230194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd5267-aaa0-4c0b-95dc-f9c659311cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:11.230505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd5267-aaa0-4c0b-95dc-f9c659311cfb admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:11.232218 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd5267-aaa0-4c0b-95dc-f9c659311cfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:11.237716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd5267-aaa0-4c0b-95dc-f9c659311cfb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:11.327810 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1138/2268] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:35:11 2022] POST /identity/v3/projects => generated 371 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:35:11.330793 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99b42e-cebc-4d50-9e69-1e6b49be34ee None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:11.350654 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99b42e-cebc-4d50-9e69-1e6b49be34ee None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:11.459942 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:11.507743 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7067e34-da4f-45b6-a2b0-04aa9a5995c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:11.511033 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7067e34-da4f-45b6-a2b0-04aa9a5995c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:11.511426 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7067e34-da4f-45b6-a2b0-04aa9a5995c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:11.511801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7067e34-da4f-45b6-a2b0-04aa9a5995c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:11.512783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7067e34-da4f-45b6-a2b0-04aa9a5995c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:11.515266 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7067e34-da4f-45b6-a2b0-04aa9a5995c5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:11.574273 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1131/2269] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:35:11 2022] POST /identity/v3/users => generated 292 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:35:11.577722 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd5267-aaa0-4c0b-95dc-f9c659311cfb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:11.607292 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd5267-aaa0-4c0b-95dc-f9c659311cfb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:11.616660 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec3fcb-3e38-4fe8-9392-48c941cb319e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:11.623809 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7067e34-da4f-45b6-a2b0-04aa9a5995c5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:11.625774 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d7067e34-da4f-45b6-a2b0-04aa9a5995c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:11.631396 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec3fcb-3e38-4fe8-9392-48c941cb319e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:11.631690 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec3fcb-3e38-4fe8-9392-48c941cb319e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:11.631976 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faec3fcb-3e38-4fe8-9392-48c941cb319e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:11.632738 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faec3fcb-3e38-4fe8-9392-48c941cb319e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:11.634377 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faec3fcb-3e38-4fe8-9392-48c941cb319e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:11.662103 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7067e34-da4f-45b6-a2b0-04aa9a5995c5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:11.665679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1139/2270] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:35:11.667993 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b173c363-5aa3-4649-b3b4-81ead7ebddb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:11.669776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173c363-5aa3-4649-b3b4-81ead7ebddb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:11.670280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173c363-5aa3-4649-b3b4-81ead7ebddb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:11.670810 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173c363-5aa3-4649-b3b4-81ead7ebddb8 admin admin] PATH_INFO: `/v3/projects/b8317294c48e4cb09969d1a6518cbb4a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:11.673788 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec3fcb-3e38-4fe8-9392-48c941cb319e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:11.675546 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b173c363-5aa3-4649-b3b4-81ead7ebddb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8317294c48e4cb09969d1a6518cbb4a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:11.678027 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b173c363-5aa3-4649-b3b4-81ead7ebddb8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:11.710861 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faec3fcb-3e38-4fe8-9392-48c941cb319e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:11.724471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360cb856-d910-4d24-82aa-c47295637d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:11.727404 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cb856-d910-4d24-82aa-c47295637d42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:11.727747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cb856-d910-4d24-82aa-c47295637d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:11.728112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cb856-d910-4d24-82aa-c47295637d42 admin admin] PATH_INFO: `/v3/projects/b499b15ff154498a96a5d304ded4cb8f/users/5d8b3b503b37428380d7268b4a1e255c/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:11.768667 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360cb856-d910-4d24-82aa-c47295637d42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b499b15ff154498a96a5d304ded4cb8f, user_id=5d8b3b503b37428380d7268b4a1e255c, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:11.774118 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360cb856-d910-4d24-82aa-c47295637d42 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:11.790007 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b173c363-5aa3-4649-b3b4-81ead7ebddb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8317294c48e4cb09969d1a6518cbb4a', 'request_id': 'req-b173c363-5aa3-4649-b3b4-81ead7ebddb8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:11.802263 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1140/2271] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:35:11 2022] PUT /identity/v3/projects/b499b15ff154498a96a5d304ded4cb8f/users/5d8b3b503b37428380d7268b4a1e255c/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:11.807435 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360cb856-d910-4d24-82aa-c47295637d42 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:11.811550 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b173c363-5aa3-4649-b3b4-81ead7ebddb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8317294c48e4cb09969d1a6518cbb4a', 'request_id': 'req-b173c363-5aa3-4649-b3b4-81ead7ebddb8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:11.827905 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b173c363-5aa3-4649-b3b4-81ead7ebddb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8317294c48e4cb09969d1a6518cbb4a', 'request_id': 'req-b173c363-5aa3-4649-b3b4-81ead7ebddb8'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:11.852609 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b173c363-5aa3-4649-b3b4-81ead7ebddb8 admin admin] The token cache is being invalidate because project b8317294c48e4cb09969d1a6518cbb4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:35:11.852928 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b173c363-5aa3-4649-b3b4-81ead7ebddb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:11.854309 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b173c363-5aa3-4649-b3b4-81ead7ebddb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:11.857514 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1132/2272] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:35:11 2022] DELETE /identity/v3/projects/b8317294c48e4cb09969d1a6518cbb4a => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:11.866933 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f4773-476a-47f5-a0a8-7f562da0f19c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:11.868111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f4773-476a-47f5-a0a8-7f562da0f19c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:11.868509 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f4773-476a-47f5-a0a8-7f562da0f19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:11.909492 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-911f4773-476a-47f5-a0a8-7f562da0f19c None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:11.910945 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-911f4773-476a-47f5-a0a8-7f562da0f19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:11.958478 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360cb856-d910-4d24-82aa-c47295637d42 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:11.971713 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f187a47-b8ee-4063-a90f-54c4e64a3ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:11.972420 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f187a47-b8ee-4063-a90f-54c4e64a3ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:11.972797 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f187a47-b8ee-4063-a90f-54c4e64a3ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:11.975703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f187a47-b8ee-4063-a90f-54c4e64a3ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:11.976464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f187a47-b8ee-4063-a90f-54c4e64a3ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:11.978094 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f187a47-b8ee-4063-a90f-54c4e64a3ab0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:12.005579 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1141/2273] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:35:12.006842 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f187a47-b8ee-4063-a90f-54c4e64a3ab0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:12.011953 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5f187a47-b8ee-4063-a90f-54c4e64a3ab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:12.033540 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f187a47-b8ee-4063-a90f-54c4e64a3ab0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:12.046870 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1133/2274] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:35:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:35:12.051037 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea930920-e7aa-4b01-a66c-46c503b77683 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:12.051339 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea930920-e7aa-4b01-a66c-46c503b77683 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:12.051713 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea930920-e7aa-4b01-a66c-46c503b77683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:12.052235 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4246cd-6912-4b29-ab1d-7d01df5bdfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:12.053379 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4246cd-6912-4b29-ab1d-7d01df5bdfd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:12.053774 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4246cd-6912-4b29-ab1d-7d01df5bdfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:12.054177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4246cd-6912-4b29-ab1d-7d01df5bdfd0 admin admin] PATH_INFO: `/v3/projects/b499b15ff154498a96a5d304ded4cb8f/users/5d8b3b503b37428380d7268b4a1e255c/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:12.061337 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4246cd-6912-4b29-ab1d-7d01df5bdfd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b499b15ff154498a96a5d304ded4cb8f, user_id=5d8b3b503b37428380d7268b4a1e255c, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:12.064613 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4246cd-6912-4b29-ab1d-7d01df5bdfd0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:12.068477 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:12.082860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1142/2275] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:35:12 2022] PUT /identity/v3/projects/b499b15ff154498a96a5d304ded4cb8f/users/5d8b3b503b37428380d7268b4a1e255c/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:12.097087 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ea930920-e7aa-4b01-a66c-46c503b77683 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:12.108807 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ea930920-e7aa-4b01-a66c-46c503b77683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:12.176016 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:12.196041 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1134/2276] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:35:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:35:12.197467 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4246cd-6912-4b29-ab1d-7d01df5bdfd0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:12.214306 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4246cd-6912-4b29-ab1d-7d01df5bdfd0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:12.223598 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983ebd9-a02e-44ed-9371-13ddd608cff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:12.224374 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983ebd9-a02e-44ed-9371-13ddd608cff8 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:12.224737 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983ebd9-a02e-44ed-9371-13ddd608cff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:12.225090 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983ebd9-a02e-44ed-9371-13ddd608cff8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:12.226405 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983ebd9-a02e-44ed-9371-13ddd608cff8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:35:12.226405 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983ebd9-a02e-44ed-9371-13ddd608cff8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:12.228354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983ebd9-a02e-44ed-9371-13ddd608cff8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:12.255226 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1143/2277] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:35:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:35:12.256482 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea930920-e7aa-4b01-a66c-46c503b77683 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:12.267640 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea930920-e7aa-4b01-a66c-46c503b77683 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:12.274075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce61e8-e48e-4d26-b394-12673a9e67fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:12.274960 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce61e8-e48e-4d26-b394-12673a9e67fe admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:12.275463 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce61e8-e48e-4d26-b394-12673a9e67fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:12.275879 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce61e8-e48e-4d26-b394-12673a9e67fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:12.276693 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ce61e8-e48e-4d26-b394-12673a9e67fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:12.284772 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ce61e8-e48e-4d26-b394-12673a9e67fe admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:12.362527 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1135/2278] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:35:12 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:35:12.366743 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983ebd9-a02e-44ed-9371-13ddd608cff8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:12.380619 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983ebd9-a02e-44ed-9371-13ddd608cff8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:12.521391 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38174b9-9380-4189-8714-719b2ac4d48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:12.523387 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38174b9-9380-4189-8714-719b2ac4d48a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:12.523914 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38174b9-9380-4189-8714-719b2ac4d48a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:12.524339 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38174b9-9380-4189-8714-719b2ac4d48a admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:12.525413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38174b9-9380-4189-8714-719b2ac4d48a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:12.527592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38174b9-9380-4189-8714-719b2ac4d48a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:12.556111 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1144/2279] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:35:12 2022] POST /identity/v3/users => generated 292 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:35:12.560167 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce61e8-e48e-4d26-b394-12673a9e67fe admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:12.581989 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce61e8-e48e-4d26-b394-12673a9e67fe admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:12.588794 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da7aa0-00ec-4415-a249-fd46c955995e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:12.589875 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da7aa0-00ec-4415-a249-fd46c955995e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:12.590270 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da7aa0-00ec-4415-a249-fd46c955995e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:12.590686 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da7aa0-00ec-4415-a249-fd46c955995e admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:12.591683 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9da7aa0-00ec-4415-a249-fd46c955995e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:12.593796 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9da7aa0-00ec-4415-a249-fd46c955995e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:12.616292 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1136/2280] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:35:12.619800 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38174b9-9380-4189-8714-719b2ac4d48a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:12.640680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38174b9-9380-4189-8714-719b2ac4d48a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:12.647624 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480c2c28-284d-4bec-971e-e532663a63d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:12.648452 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c2c28-284d-4bec-971e-e532663a63d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:12.648831 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c2c28-284d-4bec-971e-e532663a63d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:12.649510 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480c2c28-284d-4bec-971e-e532663a63d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:12.650844 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c2c28-284d-4bec-971e-e532663a63d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:12.652922 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480c2c28-284d-4bec-971e-e532663a63d4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:12.690664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1145/2281] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:35:12.697211 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da7aa0-00ec-4415-a249-fd46c955995e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:12.710261 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da7aa0-00ec-4415-a249-fd46c955995e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:12.723680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50db86d-ea49-4293-b9d5-281df0704d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:12.725322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50db86d-ea49-4293-b9d5-281df0704d77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:12.726152 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50db86d-ea49-4293-b9d5-281df0704d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:12.727060 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50db86d-ea49-4293-b9d5-281df0704d77 admin admin] PATH_INFO: `/v3/projects/f95e042c026e4aa98d8e1363247b590e/users/5113e3b17ee34ba6ac6914cd60816fec/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:12.779088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50db86d-ea49-4293-b9d5-281df0704d77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f95e042c026e4aa98d8e1363247b590e, user_id=5113e3b17ee34ba6ac6914cd60816fec, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:12.783244 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50db86d-ea49-4293-b9d5-281df0704d77 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:12.818073 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1137/2282] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:35:12 2022] PUT /identity/v3/projects/f95e042c026e4aa98d8e1363247b590e/users/5113e3b17ee34ba6ac6914cd60816fec/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:12.821416 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30999f-2eaf-4e91-bc57-1498ad13ecec None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:12.822051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30999f-2eaf-4e91-bc57-1498ad13ecec None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:12.822660 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e30999f-2eaf-4e91-bc57-1498ad13ecec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:12.922870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3e30999f-2eaf-4e91-bc57-1498ad13ecec None None] MFA Rules not processed for user `5113e3b17ee34ba6ac6914cd60816fec`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:12.925966 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3e30999f-2eaf-4e91-bc57-1498ad13ecec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:13.109975 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1146/2283] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:35:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:35:13.128593 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50db86d-ea49-4293-b9d5-281df0704d77 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:13.133859 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c50db86d-ea49-4293-b9d5-281df0704d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:13.281017 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50db86d-ea49-4293-b9d5-281df0704d77 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:13.290330 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea06f9-28c7-4bb9-8b9b-92d1b898254c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:13.291959 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea06f9-28c7-4bb9-8b9b-92d1b898254c service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:13.291959 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea06f9-28c7-4bb9-8b9b-92d1b898254c service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:13.291959 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea06f9-28c7-4bb9-8b9b-92d1b898254c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:13.304348 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea06f9-28c7-4bb9-8b9b-92d1b898254c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:13.306412 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea06f9-28c7-4bb9-8b9b-92d1b898254c service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:13.326844 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1138/2284] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:35:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:35:19.252832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0928a9-a3b4-4ff1-b6da-e6283187f4d1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:19.253526 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0928a9-a3b4-4ff1-b6da-e6283187f4d1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:19.253526 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0928a9-a3b4-4ff1-b6da-e6283187f4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:19.294744 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0e0928a9-a3b4-4ff1-b6da-e6283187f4d1 None None] MFA Rules not processed for user `5d8b3b503b37428380d7268b4a1e255c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:19.296266 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0e0928a9-a3b4-4ff1-b6da-e6283187f4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:19.376583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:19.426461 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1147/2285] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:35:19 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:35:19.445287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea06f9-28c7-4bb9-8b9b-92d1b898254c service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:19.447877 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-00ea06f9-28c7-4bb9-8b9b-92d1b898254c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:19.574897 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea06f9-28c7-4bb9-8b9b-92d1b898254c service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:19.582439 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f0b3d-7427-4cc4-974f-2dbbd2f475ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:19.583218 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f0b3d-7427-4cc4-974f-2dbbd2f475ba service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:19.583735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f0b3d-7427-4cc4-974f-2dbbd2f475ba service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:19.584126 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f0b3d-7427-4cc4-974f-2dbbd2f475ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:19.586078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ac1f0b3d-7427-4cc4-974f-2dbbd2f475ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:19.675578 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:19.761203 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f0b3d-7427-4cc4-974f-2dbbd2f475ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:19.763680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f0b3d-7427-4cc4-974f-2dbbd2f475ba service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:19.783808 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1139/2286] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:35:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:35:22.164015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51550f9b-d8c5-4f0c-b4ac-2d5ffa8a2016 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:22.164542 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51550f9b-d8c5-4f0c-b4ac-2d5ffa8a2016 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:22.164706 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51550f9b-d8c5-4f0c-b4ac-2d5ffa8a2016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:22.190867 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-51550f9b-d8c5-4f0c-b4ac-2d5ffa8a2016 None None] MFA Rules not processed for user `f026018d7f444519983df75eb4453284`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:22.192478 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-51550f9b-d8c5-4f0c-b4ac-2d5ffa8a2016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:22.209526 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1148/2287] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:35:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:35:23.353171 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f0b3d-7427-4cc4-974f-2dbbd2f475ba service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:23.459316 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f0b3d-7427-4cc4-974f-2dbbd2f475ba service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:23.464235 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be8377-1d81-4e23-9811-5ad57038cd6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:23.465257 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be8377-1d81-4e23-9811-5ad57038cd6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:23.465416 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be8377-1d81-4e23-9811-5ad57038cd6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:23.465670 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be8377-1d81-4e23-9811-5ad57038cd6b admin admin] PATH_INFO: `/v3/users/5113e3b17ee34ba6ac6914cd60816fec` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:23.466916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be8377-1d81-4e23-9811-5ad57038cd6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5113e3b17ee34ba6ac6914cd60816fec)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:23.468883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be8377-1d81-4e23-9811-5ad57038cd6b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:23.544225 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-54be8377-1d81-4e23-9811-5ad57038cd6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5113e3b17ee34ba6ac6914cd60816fec'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:23.555042 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-54be8377-1d81-4e23-9811-5ad57038cd6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5113e3b17ee34ba6ac6914cd60816fec'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:23.571995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-54be8377-1d81-4e23-9811-5ad57038cd6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5113e3b17ee34ba6ac6914cd60816fec'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:23.573020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-54be8377-1d81-4e23-9811-5ad57038cd6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5113e3b17ee34ba6ac6914cd60816fec'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:23.573835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-54be8377-1d81-4e23-9811-5ad57038cd6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5113e3b17ee34ba6ac6914cd60816fec'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:23.590474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1140/2288] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:35:23 2022] DELETE /identity/v3/users/5113e3b17ee34ba6ac6914cd60816fec => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:23.592987 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25283542-1b05-4fdf-abf6-43aca02db399 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:23.593839 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25283542-1b05-4fdf-abf6-43aca02db399 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:23.594264 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25283542-1b05-4fdf-abf6-43aca02db399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:23.614941 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-25283542-1b05-4fdf-abf6-43aca02db399 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:23.616559 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-25283542-1b05-4fdf-abf6-43aca02db399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:23.705882 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1149/2289] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:35:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:35:23.718014 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be8377-1d81-4e23-9811-5ad57038cd6b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:23.721099 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-54be8377-1d81-4e23-9811-5ad57038cd6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:23.796271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be8377-1d81-4e23-9811-5ad57038cd6b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:23.800769 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81a3d2-1403-49c4-b1ce-5a5b1beecb9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:23.801437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81a3d2-1403-49c4-b1ce-5a5b1beecb9f service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:23.801724 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81a3d2-1403-49c4-b1ce-5a5b1beecb9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:23.802056 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81a3d2-1403-49c4-b1ce-5a5b1beecb9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:23.813064 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81a3d2-1403-49c4-b1ce-5a5b1beecb9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:23.815395 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81a3d2-1403-49c4-b1ce-5a5b1beecb9f service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:23.822956 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:23.838624 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1141/2290] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:35:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:35:24.470473 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25283542-1b05-4fdf-abf6-43aca02db399 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:24.472897 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-25283542-1b05-4fdf-abf6-43aca02db399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:24.490202 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25283542-1b05-4fdf-abf6-43aca02db399 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:24.498386 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be75d0c-6184-4d22-b315-39538dd55005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:24.499837 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be75d0c-6184-4d22-b315-39538dd55005 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:24.500672 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be75d0c-6184-4d22-b315-39538dd55005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:24.502456 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be75d0c-6184-4d22-b315-39538dd55005 admin admin] PATH_INFO: `/v3/projects/f95e042c026e4aa98d8e1363247b590e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:24.504707 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be75d0c-6184-4d22-b315-39538dd55005 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f95e042c026e4aa98d8e1363247b590e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:24.507855 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be75d0c-6184-4d22-b315-39538dd55005 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:24.613151 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5be75d0c-6184-4d22-b315-39538dd55005 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f95e042c026e4aa98d8e1363247b590e', 'request_id': 'req-5be75d0c-6184-4d22-b315-39538dd55005'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:24.624402 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5be75d0c-6184-4d22-b315-39538dd55005 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f95e042c026e4aa98d8e1363247b590e', 'request_id': 'req-5be75d0c-6184-4d22-b315-39538dd55005'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:24.636249 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5be75d0c-6184-4d22-b315-39538dd55005 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f95e042c026e4aa98d8e1363247b590e', 'request_id': 'req-5be75d0c-6184-4d22-b315-39538dd55005'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:24.651095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5be75d0c-6184-4d22-b315-39538dd55005 admin admin] The token cache is being invalidate because project f95e042c026e4aa98d8e1363247b590e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:35:24.651400 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5be75d0c-6184-4d22-b315-39538dd55005 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:24.652361 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5be75d0c-6184-4d22-b315-39538dd55005 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:24.659411 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1150/2291] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:35:24 2022] DELETE /identity/v3/projects/f95e042c026e4aa98d8e1363247b590e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:24.660420 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017db9f1-0284-4d92-b400-0e884ba2009f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:24.660903 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017db9f1-0284-4d92-b400-0e884ba2009f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:24.660995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017db9f1-0284-4d92-b400-0e884ba2009f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:24.693839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-017db9f1-0284-4d92-b400-0e884ba2009f None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:24.695105 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-017db9f1-0284-4d92-b400-0e884ba2009f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:24.851353 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1142/2292] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:35:24.858950 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45e6af-1438-4985-aae8-51159bcd4381 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:24.859486 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45e6af-1438-4985-aae8-51159bcd4381 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:24.860002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45e6af-1438-4985-aae8-51159bcd4381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:24.883248 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dd45e6af-1438-4985-aae8-51159bcd4381 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:24.884766 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dd45e6af-1438-4985-aae8-51159bcd4381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:24.898262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1151/2293] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:35:24.899529 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017db9f1-0284-4d92-b400-0e884ba2009f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:24.914962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017db9f1-0284-4d92-b400-0e884ba2009f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:24.920975 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d92d48-1522-40c9-a23d-4ec714862076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:24.921660 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d92d48-1522-40c9-a23d-4ec714862076 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:24.921951 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d92d48-1522-40c9-a23d-4ec714862076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:24.922234 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d92d48-1522-40c9-a23d-4ec714862076 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:24.923620 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d92d48-1522-40c9-a23d-4ec714862076 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:35:24.924108 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d92d48-1522-40c9-a23d-4ec714862076 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:24.926038 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d92d48-1522-40c9-a23d-4ec714862076 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:24.961362 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1143/2294] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:35:24.965568 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45e6af-1438-4985-aae8-51159bcd4381 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:24.978961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45e6af-1438-4985-aae8-51159bcd4381 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:24.987699 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3636959-1f12-42ab-9314-cdc0d8144587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:24.988797 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3636959-1f12-42ab-9314-cdc0d8144587 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:24.991343 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3636959-1f12-42ab-9314-cdc0d8144587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:24.991760 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3636959-1f12-42ab-9314-cdc0d8144587 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:24.992289 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3636959-1f12-42ab-9314-cdc0d8144587 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:24.994232 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3636959-1f12-42ab-9314-cdc0d8144587 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:25.056324 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1152/2295] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:35:24 2022] POST /identity/v3/projects => generated 361 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:35:25.060220 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d92d48-1522-40c9-a23d-4ec714862076 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:25.081048 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d92d48-1522-40c9-a23d-4ec714862076 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:25.261791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068dd76-d26d-42b1-babb-1bd2c2faaf2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:25.263720 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068dd76-d26d-42b1-babb-1bd2c2faaf2c admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:25.264019 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068dd76-d26d-42b1-babb-1bd2c2faaf2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:25.264305 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068dd76-d26d-42b1-babb-1bd2c2faaf2c admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:25.270947 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068dd76-d26d-42b1-babb-1bd2c2faaf2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:25.273050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068dd76-d26d-42b1-babb-1bd2c2faaf2c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:25.327557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1144/2296] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:35:25 2022] POST /identity/v3/users => generated 281 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:35:25.327964 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3636959-1f12-42ab-9314-cdc0d8144587 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:25.342079 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3636959-1f12-42ab-9314-cdc0d8144587 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:25.351781 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8e2df-43be-4a92-b3af-df3bf3d4f9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:25.352599 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8e2df-43be-4a92-b3af-df3bf3d4f9b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:25.353003 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8e2df-43be-4a92-b3af-df3bf3d4f9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:25.353391 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8e2df-43be-4a92-b3af-df3bf3d4f9b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:25.354280 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff8e2df-43be-4a92-b3af-df3bf3d4f9b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:25.356148 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff8e2df-43be-4a92-b3af-df3bf3d4f9b9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:25.388490 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1153/2297] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:35:25.389420 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068dd76-d26d-42b1-babb-1bd2c2faaf2c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:25.417067 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068dd76-d26d-42b1-babb-1bd2c2faaf2c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:25.432551 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce47d2-bb03-4b5e-8acb-4fe1ad0936e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:25.433752 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce47d2-bb03-4b5e-8acb-4fe1ad0936e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:25.433752 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce47d2-bb03-4b5e-8acb-4fe1ad0936e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:25.434154 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce47d2-bb03-4b5e-8acb-4fe1ad0936e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:25.435022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce47d2-bb03-4b5e-8acb-4fe1ad0936e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:25.437836 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce47d2-bb03-4b5e-8acb-4fe1ad0936e2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:25.473437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1145/2298] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:35:25.475913 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8e2df-43be-4a92-b3af-df3bf3d4f9b9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:25.487888 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff8e2df-43be-4a92-b3af-df3bf3d4f9b9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:25.515302 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2ed58-1c33-4082-940c-0bc673d47de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:25.517185 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2ed58-1c33-4082-940c-0bc673d47de9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:25.517508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2ed58-1c33-4082-940c-0bc673d47de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:25.518006 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad2ed58-1c33-4082-940c-0bc673d47de9 admin admin] PATH_INFO: `/v3/projects/8b05b4f43795411ba254811fbaa9a3d0/users/c21a1ae70d744283b418d131e6007c95/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:25.530910 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:25.569745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2ed58-1c33-4082-940c-0bc673d47de9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b05b4f43795411ba254811fbaa9a3d0, user_id=c21a1ae70d744283b418d131e6007c95, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:25.573178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad2ed58-1c33-4082-940c-0bc673d47de9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:25.602869 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1154/2299] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:35:25 2022] PUT /identity/v3/projects/8b05b4f43795411ba254811fbaa9a3d0/users/c21a1ae70d744283b418d131e6007c95/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:25.603583 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f3ab9-307e-49b5-a4f5-bfd9ed3de8fb None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:25.604131 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f3ab9-307e-49b5-a4f5-bfd9ed3de8fb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:25.604521 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f3ab9-307e-49b5-a4f5-bfd9ed3de8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:25.695780 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-888f3ab9-307e-49b5-a4f5-bfd9ed3de8fb None None] MFA Rules not processed for user `c21a1ae70d744283b418d131e6007c95`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:25.698328 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-888f3ab9-307e-49b5-a4f5-bfd9ed3de8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:25.840604 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:25.861813 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1146/2300] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:35:25 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:35:25.911284 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2ed58-1c33-4082-940c-0bc673d47de9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:25.913430 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8ad2ed58-1c33-4082-940c-0bc673d47de9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:26.011118 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad2ed58-1c33-4082-940c-0bc673d47de9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:26.015501 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32a61b-06a9-4234-bf35-33f41564ceb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:26.017087 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32a61b-06a9-4234-bf35-33f41564ceb8 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:26.017670 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32a61b-06a9-4234-bf35-33f41564ceb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:26.018094 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32a61b-06a9-4234-bf35-33f41564ceb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:26.043775 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32a61b-06a9-4234-bf35-33f41564ceb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:26.045350 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32a61b-06a9-4234-bf35-33f41564ceb8 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:26.057068 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1155/2301] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:35:25 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:35:26.075740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f3ab9-307e-49b5-a4f5-bfd9ed3de8fb None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:26.209356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f3ab9-307e-49b5-a4f5-bfd9ed3de8fb None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:26.218480 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabff14d-8bf8-4f0b-8e26-b82ce601126e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:26.220230 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabff14d-8bf8-4f0b-8e26-b82ce601126e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:26.220758 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabff14d-8bf8-4f0b-8e26-b82ce601126e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:26.221185 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabff14d-8bf8-4f0b-8e26-b82ce601126e admin admin] PATH_INFO: `/v3/users/c21a1ae70d744283b418d131e6007c95` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:26.222696 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabff14d-8bf8-4f0b-8e26-b82ce601126e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c21a1ae70d744283b418d131e6007c95)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:26.227825 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabff14d-8bf8-4f0b-8e26-b82ce601126e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:26.375213 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fabff14d-8bf8-4f0b-8e26-b82ce601126e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c21a1ae70d744283b418d131e6007c95'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:26.388008 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fabff14d-8bf8-4f0b-8e26-b82ce601126e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c21a1ae70d744283b418d131e6007c95'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:26.425112 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fabff14d-8bf8-4f0b-8e26-b82ce601126e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c21a1ae70d744283b418d131e6007c95'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:26.426128 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fabff14d-8bf8-4f0b-8e26-b82ce601126e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c21a1ae70d744283b418d131e6007c95'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:26.426933 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fabff14d-8bf8-4f0b-8e26-b82ce601126e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c21a1ae70d744283b418d131e6007c95'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:26.445687 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1147/2302] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:35:26 2022] DELETE /identity/v3/users/c21a1ae70d744283b418d131e6007c95 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:26.451550 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffdcb6-924d-438e-bcaa-cc03b1ddcdd2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:26.452145 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffdcb6-924d-438e-bcaa-cc03b1ddcdd2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:26.452501 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffdcb6-924d-438e-bcaa-cc03b1ddcdd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:26.504230 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2bffdcb6-924d-438e-bcaa-cc03b1ddcdd2 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:26.505581 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2bffdcb6-924d-438e-bcaa-cc03b1ddcdd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:26.636197 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1156/2303] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:35:26.658901 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabff14d-8bf8-4f0b-8e26-b82ce601126e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:26.665999 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fabff14d-8bf8-4f0b-8e26-b82ce601126e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:26.759127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabff14d-8bf8-4f0b-8e26-b82ce601126e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:26.763742 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a4498-363f-4651-9ae9-3c70047882c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:26.767867 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a4498-363f-4651-9ae9-3c70047882c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:26.769377 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a4498-363f-4651-9ae9-3c70047882c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:26.769377 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a4498-363f-4651-9ae9-3c70047882c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:26.795491 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a4498-363f-4651-9ae9-3c70047882c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:26.797028 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a4498-363f-4651-9ae9-3c70047882c6 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:26.816655 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1148/2304] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:35:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:35:27.318606 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffdcb6-924d-438e-bcaa-cc03b1ddcdd2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:27.320198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2bffdcb6-924d-438e-bcaa-cc03b1ddcdd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:27.341943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffdcb6-924d-438e-bcaa-cc03b1ddcdd2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:27.351309 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5304f3a3-3b9f-4670-a32f-1d22fbd71910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:27.352444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304f3a3-3b9f-4670-a32f-1d22fbd71910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:27.370346 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304f3a3-3b9f-4670-a32f-1d22fbd71910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:27.370808 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5304f3a3-3b9f-4670-a32f-1d22fbd71910 admin admin] PATH_INFO: `/v3/projects/8b05b4f43795411ba254811fbaa9a3d0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:27.372409 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304f3a3-3b9f-4670-a32f-1d22fbd71910 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b05b4f43795411ba254811fbaa9a3d0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:27.374591 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5304f3a3-3b9f-4670-a32f-1d22fbd71910 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:27.432808 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5304f3a3-3b9f-4670-a32f-1d22fbd71910 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b05b4f43795411ba254811fbaa9a3d0', 'request_id': 'req-5304f3a3-3b9f-4670-a32f-1d22fbd71910'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:27.441363 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5304f3a3-3b9f-4670-a32f-1d22fbd71910 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b05b4f43795411ba254811fbaa9a3d0', 'request_id': 'req-5304f3a3-3b9f-4670-a32f-1d22fbd71910'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:27.448781 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5304f3a3-3b9f-4670-a32f-1d22fbd71910 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b05b4f43795411ba254811fbaa9a3d0', 'request_id': 'req-5304f3a3-3b9f-4670-a32f-1d22fbd71910'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:27.460746 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5304f3a3-3b9f-4670-a32f-1d22fbd71910 admin admin] The token cache is being invalidate because project 8b05b4f43795411ba254811fbaa9a3d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:35:27.461036 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5304f3a3-3b9f-4670-a32f-1d22fbd71910 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:27.461814 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5304f3a3-3b9f-4670-a32f-1d22fbd71910 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:27.463619 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1157/2305] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:35:27 2022] DELETE /identity/v3/projects/8b05b4f43795411ba254811fbaa9a3d0 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:27.468674 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087f2fd-215d-435e-b67b-f285c7cdadfc None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:27.469249 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087f2fd-215d-435e-b67b-f285c7cdadfc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:27.470664 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087f2fd-215d-435e-b67b-f285c7cdadfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:27.490783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b087f2fd-215d-435e-b67b-f285c7cdadfc None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:27.492055 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b087f2fd-215d-435e-b67b-f285c7cdadfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:27.504915 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:27.568789 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1149/2306] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:35:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:35:27.579338 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43564013-bf4d-4dcf-8482-512ee9dcfc1d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:27.580377 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43564013-bf4d-4dcf-8482-512ee9dcfc1d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:27.580920 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43564013-bf4d-4dcf-8482-512ee9dcfc1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:27.607109 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-43564013-bf4d-4dcf-8482-512ee9dcfc1d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:27.608621 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-43564013-bf4d-4dcf-8482-512ee9dcfc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:27.618535 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1158/2307] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:35:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:35:27.622827 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087f2fd-215d-435e-b67b-f285c7cdadfc None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:27.635155 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087f2fd-215d-435e-b67b-f285c7cdadfc None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:27.641141 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95781cef-4f0e-4775-bb5e-64fa4ba4c391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:27.642428 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95781cef-4f0e-4775-bb5e-64fa4ba4c391 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:27.642857 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95781cef-4f0e-4775-bb5e-64fa4ba4c391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:27.643256 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95781cef-4f0e-4775-bb5e-64fa4ba4c391 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:27.644029 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95781cef-4f0e-4775-bb5e-64fa4ba4c391 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:35:27.644463 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95781cef-4f0e-4775-bb5e-64fa4ba4c391 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:27.646499 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95781cef-4f0e-4775-bb5e-64fa4ba4c391 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:27.671403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1150/2308] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:35:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:35:27.672490 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43564013-bf4d-4dcf-8482-512ee9dcfc1d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:27.684841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43564013-bf4d-4dcf-8482-512ee9dcfc1d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:27.691529 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ff9f9-4644-437b-a764-cfda732ee223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:27.692196 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ff9f9-4644-437b-a764-cfda732ee223 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:27.692643 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ff9f9-4644-437b-a764-cfda732ee223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:27.693109 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ff9f9-4644-437b-a764-cfda732ee223 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:27.693810 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7ff9f9-4644-437b-a764-cfda732ee223 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:27.695733 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7ff9f9-4644-437b-a764-cfda732ee223 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:27.699424 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:27.730978 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1159/2309] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:35:27 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:35:27.735771 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95781cef-4f0e-4775-bb5e-64fa4ba4c391 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:27.746695 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95781cef-4f0e-4775-bb5e-64fa4ba4c391 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:27.812091 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7bade-49d7-4300-a569-3131f48943e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:27.813821 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7bade-49d7-4300-a569-3131f48943e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:27.814117 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7bade-49d7-4300-a569-3131f48943e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:27.814402 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c7bade-49d7-4300-a569-3131f48943e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:27.815354 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7bade-49d7-4300-a569-3131f48943e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:27.817303 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c7bade-49d7-4300-a569-3131f48943e7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:27.841720 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1151/2310] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:35:27 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:35:27.843005 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ff9f9-4644-437b-a764-cfda732ee223 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:27.854915 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ff9f9-4644-437b-a764-cfda732ee223 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:27.861617 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a64f3c-b030-4ae1-b393-f3348c7366ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:27.862529 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a64f3c-b030-4ae1-b393-f3348c7366ec admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:27.862982 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a64f3c-b030-4ae1-b393-f3348c7366ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:27.863267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a64f3c-b030-4ae1-b393-f3348c7366ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:27.864010 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a64f3c-b030-4ae1-b393-f3348c7366ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:27.865619 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a64f3c-b030-4ae1-b393-f3348c7366ec admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:27.877643 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1160/2311] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:35:27.881756 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7bade-49d7-4300-a569-3131f48943e7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:27.893134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c7bade-49d7-4300-a569-3131f48943e7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:27.898353 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef811b-e531-4b84-9f74-bd43ff5f69ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:27.899076 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef811b-e531-4b84-9f74-bd43ff5f69ed admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:27.899652 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef811b-e531-4b84-9f74-bd43ff5f69ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:27.899801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef811b-e531-4b84-9f74-bd43ff5f69ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:27.900775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef811b-e531-4b84-9f74-bd43ff5f69ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:27.902555 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef811b-e531-4b84-9f74-bd43ff5f69ed admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:27.914472 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1152/2312] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:35:27.918471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a64f3c-b030-4ae1-b393-f3348c7366ec admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:27.930828 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a64f3c-b030-4ae1-b393-f3348c7366ec admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:27.935774 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448785f-7c05-4d8c-85a7-6564ad855497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:27.936936 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448785f-7c05-4d8c-85a7-6564ad855497 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:27.937462 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448785f-7c05-4d8c-85a7-6564ad855497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:27.938006 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448785f-7c05-4d8c-85a7-6564ad855497 admin admin] PATH_INFO: `/v3/projects/bb38a5a07359440a8b15656b5d202899/users/ecb42a0158e842baae6a2ab5d99710e4/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:27.961734 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3448785f-7c05-4d8c-85a7-6564ad855497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb38a5a07359440a8b15656b5d202899, user_id=ecb42a0158e842baae6a2ab5d99710e4, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:27.965313 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3448785f-7c05-4d8c-85a7-6564ad855497 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:27.983863 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1161/2313] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:35:27 2022] PUT /identity/v3/projects/bb38a5a07359440a8b15656b5d202899/users/ecb42a0158e842baae6a2ab5d99710e4/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:27.995736 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea8198-aada-480e-9e32-22ed3998ae35 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:27.996410 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea8198-aada-480e-9e32-22ed3998ae35 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:27.996855 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea8198-aada-480e-9e32-22ed3998ae35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:28.031438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-53ea8198-aada-480e-9e32-22ed3998ae35 None None] MFA Rules not processed for user `ecb42a0158e842baae6a2ab5d99710e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:28.033473 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-53ea8198-aada-480e-9e32-22ed3998ae35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:28.110914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1153/2314] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:35:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:35:28.111976 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448785f-7c05-4d8c-85a7-6564ad855497 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:28.182764 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448785f-7c05-4d8c-85a7-6564ad855497 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:28.189059 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8fbff1-a844-4649-86b2-2502e670318d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:28.189724 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8fbff1-a844-4649-86b2-2502e670318d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:28.190036 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8fbff1-a844-4649-86b2-2502e670318d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:28.190322 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8fbff1-a844-4649-86b2-2502e670318d admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:28.191189 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8fbff1-a844-4649-86b2-2502e670318d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:28.193621 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8fbff1-a844-4649-86b2-2502e670318d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:28.267513 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1162/2315] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:35:28 2022] POST /identity/v3/projects => generated 375 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:35:28.274150 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea8198-aada-480e-9e32-22ed3998ae35 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:28.302605 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea8198-aada-480e-9e32-22ed3998ae35 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:28.313816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:28.406339 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677cb6b-d5b7-43ed-bfde-f918f1ef76db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:28.407526 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677cb6b-d5b7-43ed-bfde-f918f1ef76db admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:28.407874 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677cb6b-d5b7-43ed-bfde-f918f1ef76db admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:28.408081 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677cb6b-d5b7-43ed-bfde-f918f1ef76db admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:28.408754 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b677cb6b-d5b7-43ed-bfde-f918f1ef76db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:28.410829 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b677cb6b-d5b7-43ed-bfde-f918f1ef76db admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:28.434489 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1154/2316] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:35:28 2022] POST /identity/v3/users => generated 294 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:35:28.439495 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8fbff1-a844-4649-86b2-2502e670318d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:28.452156 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8fbff1-a844-4649-86b2-2502e670318d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:28.459121 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a55e6-d3b3-44d6-8e6e-fa606bcd85ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:28.460639 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a55e6-d3b3-44d6-8e6e-fa606bcd85ea admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:28.460762 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a55e6-d3b3-44d6-8e6e-fa606bcd85ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:28.461627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a55e6-d3b3-44d6-8e6e-fa606bcd85ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:28.462578 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a55e6-d3b3-44d6-8e6e-fa606bcd85ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:28.464257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a55e6-d3b3-44d6-8e6e-fa606bcd85ea admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:28.482009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1163/2317] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:35:28.483301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677cb6b-d5b7-43ed-bfde-f918f1ef76db admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:28.495784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677cb6b-d5b7-43ed-bfde-f918f1ef76db admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:28.501134 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a938fe0-a849-4ea4-a06b-a44f29abea6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:28.502290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a938fe0-a849-4ea4-a06b-a44f29abea6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:28.502627 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a938fe0-a849-4ea4-a06b-a44f29abea6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:28.502991 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a938fe0-a849-4ea4-a06b-a44f29abea6d admin admin] PATH_INFO: `/v3/projects/860c43c0995e45f9a766ea6b614e3237/users/bb294429c76041e8a079a4631169e8ad/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:28.519124 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a938fe0-a849-4ea4-a06b-a44f29abea6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=860c43c0995e45f9a766ea6b614e3237, user_id=bb294429c76041e8a079a4631169e8ad, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:28.522122 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a938fe0-a849-4ea4-a06b-a44f29abea6d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:28.537914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1155/2318] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:35:28 2022] PUT /identity/v3/projects/860c43c0995e45f9a766ea6b614e3237/users/bb294429c76041e8a079a4631169e8ad/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:28.542037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a55e6-d3b3-44d6-8e6e-fa606bcd85ea admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:28.555084 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a55e6-d3b3-44d6-8e6e-fa606bcd85ea admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:28.640099 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce714c-354d-4b4a-b230-1e0a43272b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:28.642861 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce714c-354d-4b4a-b230-1e0a43272b95 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:28.643156 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce714c-354d-4b4a-b230-1e0a43272b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:28.643434 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce714c-354d-4b4a-b230-1e0a43272b95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:28.644181 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce714c-354d-4b4a-b230-1e0a43272b95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:28.646148 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce714c-354d-4b4a-b230-1e0a43272b95 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:28.662170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1164/2319] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:35:28.669005 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a938fe0-a849-4ea4-a06b-a44f29abea6d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:28.687167 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a938fe0-a849-4ea4-a06b-a44f29abea6d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:28.698604 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790e2e6-71a6-45ca-a765-16a41df17c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:28.700081 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790e2e6-71a6-45ca-a765-16a41df17c5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:28.700476 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790e2e6-71a6-45ca-a765-16a41df17c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:28.700893 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790e2e6-71a6-45ca-a765-16a41df17c5e admin admin] PATH_INFO: `/v3/projects/860c43c0995e45f9a766ea6b614e3237/users/bb294429c76041e8a079a4631169e8ad/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:28.704584 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0790e2e6-71a6-45ca-a765-16a41df17c5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=860c43c0995e45f9a766ea6b614e3237, user_id=bb294429c76041e8a079a4631169e8ad, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:28.708529 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0790e2e6-71a6-45ca-a765-16a41df17c5e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:28.724770 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1156/2320] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:35:28 2022] PUT /identity/v3/projects/860c43c0995e45f9a766ea6b614e3237/users/bb294429c76041e8a079a4631169e8ad/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:28.739109 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b376c42-44b6-4384-bc6e-372da1a5dd71 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:28.744427 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b376c42-44b6-4384-bc6e-372da1a5dd71 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:28.744824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b376c42-44b6-4384-bc6e-372da1a5dd71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:28.786274 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1b376c42-44b6-4384-bc6e-372da1a5dd71 None None] MFA Rules not processed for user `bb294429c76041e8a079a4631169e8ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:28.788604 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1b376c42-44b6-4384-bc6e-372da1a5dd71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:28.799460 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:28.881067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1165/2321] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:35:28 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:35:28.912113 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790e2e6-71a6-45ca-a765-16a41df17c5e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:28.914370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0790e2e6-71a6-45ca-a765-16a41df17c5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:29.013547 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790e2e6-71a6-45ca-a765-16a41df17c5e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:29.019288 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f68489-1138-442b-a5bd-b6e10f5ede6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:29.022221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f68489-1138-442b-a5bd-b6e10f5ede6b service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:29.022772 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f68489-1138-442b-a5bd-b6e10f5ede6b service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:29.023361 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f68489-1138-442b-a5bd-b6e10f5ede6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:29.106934 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f68489-1138-442b-a5bd-b6e10f5ede6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:29.108683 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f68489-1138-442b-a5bd-b6e10f5ede6b service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:29.132565 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1157/2322] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:35:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:35:32.148412 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b376c42-44b6-4384-bc6e-372da1a5dd71 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:32.223278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b376c42-44b6-4384-bc6e-372da1a5dd71 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:32.241417 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691c206f-d763-4e5e-87d4-3b2d1ba4d993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:32.241417 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c206f-d763-4e5e-87d4-3b2d1ba4d993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:32.241417 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c206f-d763-4e5e-87d4-3b2d1ba4d993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:32.241417 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c206f-d763-4e5e-87d4-3b2d1ba4d993 admin admin] PATH_INFO: `/v3/users/ecb42a0158e842baae6a2ab5d99710e4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:32.245469 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691c206f-d763-4e5e-87d4-3b2d1ba4d993 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecb42a0158e842baae6a2ab5d99710e4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:32.247626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691c206f-d763-4e5e-87d4-3b2d1ba4d993 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:32.322241 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-691c206f-d763-4e5e-87d4-3b2d1ba4d993 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecb42a0158e842baae6a2ab5d99710e4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:32.323129 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-691c206f-d763-4e5e-87d4-3b2d1ba4d993 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecb42a0158e842baae6a2ab5d99710e4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:32.336149 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-691c206f-d763-4e5e-87d4-3b2d1ba4d993 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecb42a0158e842baae6a2ab5d99710e4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:32.336857 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-691c206f-d763-4e5e-87d4-3b2d1ba4d993 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecb42a0158e842baae6a2ab5d99710e4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:32.343837 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-691c206f-d763-4e5e-87d4-3b2d1ba4d993 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecb42a0158e842baae6a2ab5d99710e4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:32.358334 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1166/2323] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:35:32 2022] DELETE /identity/v3/users/ecb42a0158e842baae6a2ab5d99710e4 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:32.363328 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df866731-0f21-43a2-8d02-27ee3109d073 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:32.363860 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df866731-0f21-43a2-8d02-27ee3109d073 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:32.363958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df866731-0f21-43a2-8d02-27ee3109d073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:32.381689 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-df866731-0f21-43a2-8d02-27ee3109d073 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:32.382923 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-df866731-0f21-43a2-8d02-27ee3109d073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:32.398610 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:32.458274 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1158/2324] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:35:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:35:32.468042 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691c206f-d763-4e5e-87d4-3b2d1ba4d993 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:32.470520 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-691c206f-d763-4e5e-87d4-3b2d1ba4d993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:32.499109 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:32.536117 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691c206f-d763-4e5e-87d4-3b2d1ba4d993 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:32.539910 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e118e6a-9cd4-4a51-96f6-5efc9f9333f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:32.540786 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e118e6a-9cd4-4a51-96f6-5efc9f9333f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:32.541209 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e118e6a-9cd4-4a51-96f6-5efc9f9333f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:32.541691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e118e6a-9cd4-4a51-96f6-5efc9f9333f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:32.553281 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e118e6a-9cd4-4a51-96f6-5efc9f9333f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:32.568536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e118e6a-9cd4-4a51-96f6-5efc9f9333f6 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:32.577617 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1167/2325] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:35:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:35:32.879947 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df866731-0f21-43a2-8d02-27ee3109d073 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:32.881455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-df866731-0f21-43a2-8d02-27ee3109d073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:32.900953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df866731-0f21-43a2-8d02-27ee3109d073 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:32.905893 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8c7610-34b5-4b83-a652-3060f1844638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:32.907690 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c7610-34b5-4b83-a652-3060f1844638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:32.907989 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c7610-34b5-4b83-a652-3060f1844638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:32.908415 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8c7610-34b5-4b83-a652-3060f1844638 admin admin] PATH_INFO: `/v3/projects/bb38a5a07359440a8b15656b5d202899` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:32.909733 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c7610-34b5-4b83-a652-3060f1844638 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb38a5a07359440a8b15656b5d202899)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:32.911841 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8c7610-34b5-4b83-a652-3060f1844638 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:32.962691 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ab8c7610-34b5-4b83-a652-3060f1844638 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb38a5a07359440a8b15656b5d202899', 'request_id': 'req-ab8c7610-34b5-4b83-a652-3060f1844638'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:32.969164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ab8c7610-34b5-4b83-a652-3060f1844638 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb38a5a07359440a8b15656b5d202899', 'request_id': 'req-ab8c7610-34b5-4b83-a652-3060f1844638'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:32.974816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ab8c7610-34b5-4b83-a652-3060f1844638 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb38a5a07359440a8b15656b5d202899', 'request_id': 'req-ab8c7610-34b5-4b83-a652-3060f1844638'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:32.988276 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ab8c7610-34b5-4b83-a652-3060f1844638 admin admin] The token cache is being invalidate because project bb38a5a07359440a8b15656b5d202899 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:35:32.988633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ab8c7610-34b5-4b83-a652-3060f1844638 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:32.989362 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ab8c7610-34b5-4b83-a652-3060f1844638 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:32.994010 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1159/2326] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:35:32 2022] DELETE /identity/v3/projects/bb38a5a07359440a8b15656b5d202899 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:32.995580 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e118e6a-9cd4-4a51-96f6-5efc9f9333f6 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:33.007722 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e118e6a-9cd4-4a51-96f6-5efc9f9333f6 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:33.067192 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d5d41-25e4-4b60-8748-71b99fba1ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:33.068147 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d5d41-25e4-4b60-8748-71b99fba1ac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:33.068439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d5d41-25e4-4b60-8748-71b99fba1ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:33.068719 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d5d41-25e4-4b60-8748-71b99fba1ac6 admin admin] PATH_INFO: `/v3/users/bb294429c76041e8a079a4631169e8ad` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:33.069907 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d5d41-25e4-4b60-8748-71b99fba1ac6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb294429c76041e8a079a4631169e8ad)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:33.071921 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d5d41-25e4-4b60-8748-71b99fba1ac6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:33.125425 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ce4d5d41-25e4-4b60-8748-71b99fba1ac6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb294429c76041e8a079a4631169e8ad'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:33.126208 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ce4d5d41-25e4-4b60-8748-71b99fba1ac6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb294429c76041e8a079a4631169e8ad'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:33.139498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ce4d5d41-25e4-4b60-8748-71b99fba1ac6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb294429c76041e8a079a4631169e8ad'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:33.140193 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ce4d5d41-25e4-4b60-8748-71b99fba1ac6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb294429c76041e8a079a4631169e8ad'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:33.147299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ce4d5d41-25e4-4b60-8748-71b99fba1ac6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb294429c76041e8a079a4631169e8ad'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:33.160196 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1168/2327] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:35:32 2022] DELETE /identity/v3/users/bb294429c76041e8a079a4631169e8ad => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:33.451649 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f207e09-f523-4d4c-97e9-3d5783847566 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:33.451649 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f207e09-f523-4d4c-97e9-3d5783847566 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:33.451649 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f207e09-f523-4d4c-97e9-3d5783847566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:33.473571 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5f207e09-f523-4d4c-97e9-3d5783847566 None None] MFA Rules not processed for user `8c10852a728844c6ae0a48e3772d95b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:33.475386 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5f207e09-f523-4d4c-97e9-3d5783847566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:33.546326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d5d41-25e4-4b60-8748-71b99fba1ac6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:33.548748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ce4d5d41-25e4-4b60-8748-71b99fba1ac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:33.552577 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1160/2328] 10.222.0.52 () {58 vars in 967 bytes} [Wed Jun 29 12:35:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:35:33.578452 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f207e09-f523-4d4c-97e9-3d5783847566 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:33.582270 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5f207e09-f523-4d4c-97e9-3d5783847566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:33.637132 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:33.651448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:33.668446 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d5d41-25e4-4b60-8748-71b99fba1ac6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:33.675215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550fd5c-5c34-43bc-89f8-18dba4ca3636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:33.676215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550fd5c-5c34-43bc-89f8-18dba4ca3636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:33.676671 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550fd5c-5c34-43bc-89f8-18dba4ca3636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:33.676963 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550fd5c-5c34-43bc-89f8-18dba4ca3636 admin admin] PATH_INFO: `/v3/projects/860c43c0995e45f9a766ea6b614e3237` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:33.678167 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8550fd5c-5c34-43bc-89f8-18dba4ca3636 admin admin] RBAC: Authorizing `identity:delete_project(project_id=860c43c0995e45f9a766ea6b614e3237)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:33.680191 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8550fd5c-5c34-43bc-89f8-18dba4ca3636 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:33.691753 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f207e09-f523-4d4c-97e9-3d5783847566 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:33.698080 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a3283-c01a-4099-b0fe-8efabb2f6794 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209b0df45f76404da880a9ae0c637d67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:33.699038 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a3283-c01a-4099-b0fe-8efabb2f6794 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:33.699452 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a3283-c01a-4099-b0fe-8efabb2f6794 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:33.699868 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a3283-c01a-4099-b0fe-8efabb2f6794 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:33.712100 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a3283-c01a-4099-b0fe-8efabb2f6794 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:33.715024 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a3283-c01a-4099-b0fe-8efabb2f6794 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:33.724292 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1161/2329] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 12:35:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 12:35:33.737137 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8550fd5c-5c34-43bc-89f8-18dba4ca3636 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '860c43c0995e45f9a766ea6b614e3237', 'request_id': 'req-8550fd5c-5c34-43bc-89f8-18dba4ca3636'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:33.744805 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8550fd5c-5c34-43bc-89f8-18dba4ca3636 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '860c43c0995e45f9a766ea6b614e3237', 'request_id': 'req-8550fd5c-5c34-43bc-89f8-18dba4ca3636'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:33.752507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8550fd5c-5c34-43bc-89f8-18dba4ca3636 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '860c43c0995e45f9a766ea6b614e3237', 'request_id': 'req-8550fd5c-5c34-43bc-89f8-18dba4ca3636'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:33.772527 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8550fd5c-5c34-43bc-89f8-18dba4ca3636 admin admin] The token cache is being invalidate because project 860c43c0995e45f9a766ea6b614e3237 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:35:33.773039 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8550fd5c-5c34-43bc-89f8-18dba4ca3636 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:33.774220 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8550fd5c-5c34-43bc-89f8-18dba4ca3636 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:33.775721 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1169/2330] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:35:33 2022] DELETE /identity/v3/projects/860c43c0995e45f9a766ea6b614e3237 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:33.780488 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19490ea-5c03-4116-a00a-07dcba677673 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:33.780776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19490ea-5c03-4116-a00a-07dcba677673 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:33.781053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19490ea-5c03-4116-a00a-07dcba677673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:33.804816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f19490ea-5c03-4116-a00a-07dcba677673 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:33.806784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f19490ea-5c03-4116-a00a-07dcba677673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:33.881501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1162/2331] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:35:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:35:33.890737 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ccaa72-bb4e-4dad-a848-9c0fbf0eaec2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:33.891812 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ccaa72-bb4e-4dad-a848-9c0fbf0eaec2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:33.892149 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ccaa72-bb4e-4dad-a848-9c0fbf0eaec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:33.912609 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-10ccaa72-bb4e-4dad-a848-9c0fbf0eaec2 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:33.913819 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-10ccaa72-bb4e-4dad-a848-9c0fbf0eaec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:33.923407 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1170/2332] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:35:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:35:33.927355 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19490ea-5c03-4116-a00a-07dcba677673 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:33.937495 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19490ea-5c03-4116-a00a-07dcba677673 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:33.946595 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac2252-a749-4f33-b543-3e6aecfc87d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:33.947477 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac2252-a749-4f33-b543-3e6aecfc87d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:33.947650 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac2252-a749-4f33-b543-3e6aecfc87d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:33.947932 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac2252-a749-4f33-b543-3e6aecfc87d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:33.948818 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac2252-a749-4f33-b543-3e6aecfc87d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:35:33.949085 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac2252-a749-4f33-b543-3e6aecfc87d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:33.950843 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac2252-a749-4f33-b543-3e6aecfc87d5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:33.966544 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1163/2333] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:35:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:35:33.969321 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ccaa72-bb4e-4dad-a848-9c0fbf0eaec2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:33.979415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ccaa72-bb4e-4dad-a848-9c0fbf0eaec2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:33.985429 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af6e50-3a70-41c9-b501-7bfd4bb400a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:33.986100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af6e50-3a70-41c9-b501-7bfd4bb400a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:33.986386 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af6e50-3a70-41c9-b501-7bfd4bb400a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:33.986731 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37af6e50-3a70-41c9-b501-7bfd4bb400a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:33.987429 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37af6e50-3a70-41c9-b501-7bfd4bb400a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:33.989338 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37af6e50-3a70-41c9-b501-7bfd4bb400a1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:34.020080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1171/2334] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:35:33 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:35:34.024823 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac2252-a749-4f33-b543-3e6aecfc87d5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:34.035105 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac2252-a749-4f33-b543-3e6aecfc87d5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:34.108562 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8af79-f36e-4905-99b6-00130d03c135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:34.109284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8af79-f36e-4905-99b6-00130d03c135 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:34.110138 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8af79-f36e-4905-99b6-00130d03c135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:34.110504 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa8af79-f36e-4905-99b6-00130d03c135 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:34.111570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa8af79-f36e-4905-99b6-00130d03c135 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:34.113563 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa8af79-f36e-4905-99b6-00130d03c135 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:34.139081 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1164/2335] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:35:34 2022] POST /identity/v3/users => generated 286 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:35:34.143902 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af6e50-3a70-41c9-b501-7bfd4bb400a1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:34.156324 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37af6e50-3a70-41c9-b501-7bfd4bb400a1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:34.163624 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df68626-61a3-46d8-ad2e-518381e8d9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:34.164387 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df68626-61a3-46d8-ad2e-518381e8d9f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:34.164743 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df68626-61a3-46d8-ad2e-518381e8d9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:34.165091 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df68626-61a3-46d8-ad2e-518381e8d9f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:34.165901 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df68626-61a3-46d8-ad2e-518381e8d9f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:34.167691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df68626-61a3-46d8-ad2e-518381e8d9f9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:34.182177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1172/2336] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:35:34.186515 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8af79-f36e-4905-99b6-00130d03c135 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:34.197801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa8af79-f36e-4905-99b6-00130d03c135 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:34.202540 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28014dec-ff70-4938-8aa4-b292acb24ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:34.203207 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28014dec-ff70-4938-8aa4-b292acb24ce7 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:34.204156 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28014dec-ff70-4938-8aa4-b292acb24ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:34.204948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28014dec-ff70-4938-8aa4-b292acb24ce7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:34.205682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28014dec-ff70-4938-8aa4-b292acb24ce7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:34.207328 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28014dec-ff70-4938-8aa4-b292acb24ce7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:34.218057 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1165/2337] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:35:34.223504 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df68626-61a3-46d8-ad2e-518381e8d9f9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:34.233752 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df68626-61a3-46d8-ad2e-518381e8d9f9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:34.238448 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d22549-78b6-4298-9bb7-ec8f65a9940e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:34.239368 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d22549-78b6-4298-9bb7-ec8f65a9940e admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:34.239680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d22549-78b6-4298-9bb7-ec8f65a9940e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:34.239963 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d22549-78b6-4298-9bb7-ec8f65a9940e admin admin] PATH_INFO: `/v3/projects/f826993a33644995a3150d71cc40add7/users/c2518c8b1d0146dcb93fd4e875389f66/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:34.255951 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d22549-78b6-4298-9bb7-ec8f65a9940e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f826993a33644995a3150d71cc40add7, user_id=c2518c8b1d0146dcb93fd4e875389f66, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:34.258717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d22549-78b6-4298-9bb7-ec8f65a9940e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:34.273872 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1173/2338] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:35:34 2022] PUT /identity/v3/projects/f826993a33644995a3150d71cc40add7/users/c2518c8b1d0146dcb93fd4e875389f66/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:34.285496 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657a01f-e762-414d-aab5-d9ebd41e9713 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:34.285871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657a01f-e762-414d-aab5-d9ebd41e9713 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:34.286178 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657a01f-e762-414d-aab5-d9ebd41e9713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:34.313538 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e657a01f-e762-414d-aab5-d9ebd41e9713 None None] MFA Rules not processed for user `c2518c8b1d0146dcb93fd4e875389f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:34.314777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e657a01f-e762-414d-aab5-d9ebd41e9713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:34.344473 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:34.386557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1166/2339] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:35:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:35:34.398907 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d22549-78b6-4298-9bb7-ec8f65a9940e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:34.400798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-42d22549-78b6-4298-9bb7-ec8f65a9940e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:34.460494 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d22549-78b6-4298-9bb7-ec8f65a9940e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:34.464233 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194466b-f22d-4b5e-9d0a-0424e504c1ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:34.464877 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194466b-f22d-4b5e-9d0a-0424e504c1ce service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:34.465161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194466b-f22d-4b5e-9d0a-0424e504c1ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:34.465438 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194466b-f22d-4b5e-9d0a-0424e504c1ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:34.474119 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4194466b-f22d-4b5e-9d0a-0424e504c1ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:34.475591 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4194466b-f22d-4b5e-9d0a-0424e504c1ce service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:34.484595 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1174/2340] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:35:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:35:35.869179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066e856-0062-4d43-aef5-3ec4f4fde5c4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:35.869179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066e856-0062-4d43-aef5-3ec4f4fde5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:35.869179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066e856-0062-4d43-aef5-3ec4f4fde5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:35.871862 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194466b-f22d-4b5e-9d0a-0424e504c1ce service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:35.877243 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4194466b-f22d-4b5e-9d0a-0424e504c1ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:35.888071 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0066e856-0062-4d43-aef5-3ec4f4fde5c4 None None] MFA Rules not processed for user `c2518c8b1d0146dcb93fd4e875389f66`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:35.889255 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0066e856-0062-4d43-aef5-3ec4f4fde5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:35.897144 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1167/2341] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:35:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:35:35.898310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:35.972361 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194466b-f22d-4b5e-9d0a-0424e504c1ce service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:35.990569 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456c2a8-a276-410f-8d4a-7605160811f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:35.991758 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456c2a8-a276-410f-8d4a-7605160811f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:35.992177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456c2a8-a276-410f-8d4a-7605160811f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:35.992539 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456c2a8-a276-410f-8d4a-7605160811f1 admin admin] PATH_INFO: `/v3/users/bba6a17fc19645c29fce6fa655095a0a` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:35.994134 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d456c2a8-a276-410f-8d4a-7605160811f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bba6a17fc19645c29fce6fa655095a0a)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:35.996607 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d456c2a8-a276-410f-8d4a-7605160811f1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:36.061507 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d456c2a8-a276-410f-8d4a-7605160811f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bba6a17fc19645c29fce6fa655095a0a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:36.062292 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d456c2a8-a276-410f-8d4a-7605160811f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bba6a17fc19645c29fce6fa655095a0a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:36.077669 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d456c2a8-a276-410f-8d4a-7605160811f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bba6a17fc19645c29fce6fa655095a0a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:36.079658 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d456c2a8-a276-410f-8d4a-7605160811f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bba6a17fc19645c29fce6fa655095a0a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:36.086940 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d456c2a8-a276-410f-8d4a-7605160811f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bba6a17fc19645c29fce6fa655095a0a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:36.103183 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1175/2342] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:35:35 2022] DELETE /identity/v3/users/bba6a17fc19645c29fce6fa655095a0a => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:36.949634 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066e856-0062-4d43-aef5-3ec4f4fde5c4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:36.962442 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066e856-0062-4d43-aef5-3ec4f4fde5c4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:37.039034 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df21ad4-7a4b-4b15-a47b-3ac305f85e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:37.039916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df21ad4-7a4b-4b15-a47b-3ac305f85e49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:37.040200 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df21ad4-7a4b-4b15-a47b-3ac305f85e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:37.040478 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df21ad4-7a4b-4b15-a47b-3ac305f85e49 admin admin] PATH_INFO: `/v3/projects/ed93385ff03f4672acc79e103ffcb82e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:37.041705 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df21ad4-7a4b-4b15-a47b-3ac305f85e49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed93385ff03f4672acc79e103ffcb82e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:37.045091 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df21ad4-7a4b-4b15-a47b-3ac305f85e49 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:37.103557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3df21ad4-7a4b-4b15-a47b-3ac305f85e49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed93385ff03f4672acc79e103ffcb82e', 'request_id': 'req-3df21ad4-7a4b-4b15-a47b-3ac305f85e49'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:37.110966 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3df21ad4-7a4b-4b15-a47b-3ac305f85e49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed93385ff03f4672acc79e103ffcb82e', 'request_id': 'req-3df21ad4-7a4b-4b15-a47b-3ac305f85e49'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:37.116725 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3df21ad4-7a4b-4b15-a47b-3ac305f85e49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed93385ff03f4672acc79e103ffcb82e', 'request_id': 'req-3df21ad4-7a4b-4b15-a47b-3ac305f85e49'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:37.131023 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3df21ad4-7a4b-4b15-a47b-3ac305f85e49 admin admin] The token cache is being invalidate because project ed93385ff03f4672acc79e103ffcb82e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:35:37.131315 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3df21ad4-7a4b-4b15-a47b-3ac305f85e49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:37.132098 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3df21ad4-7a4b-4b15-a47b-3ac305f85e49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:35:37.133644 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1168/2343] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:35:36 2022] DELETE /identity/v3/projects/ed93385ff03f4672acc79e103ffcb82e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:37.138741 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc12896-a39d-47e6-9853-95becba57f88 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:37.139234 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc12896-a39d-47e6-9853-95becba57f88 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:37.139234 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc12896-a39d-47e6-9853-95becba57f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:37.158966 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2bc12896-a39d-47e6-9853-95becba57f88 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:37.160200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2bc12896-a39d-47e6-9853-95becba57f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:37.222076 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1176/2344] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:35:37.230250 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61783247-0794-4c24-9f05-57849864e40b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:37.230703 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61783247-0794-4c24-9f05-57849864e40b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:37.230851 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61783247-0794-4c24-9f05-57849864e40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:37.249473 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-61783247-0794-4c24-9f05-57849864e40b None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:37.251471 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-61783247-0794-4c24-9f05-57849864e40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:37.259399 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1169/2345] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:35:37.263566 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc12896-a39d-47e6-9853-95becba57f88 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:37.273440 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc12896-a39d-47e6-9853-95becba57f88 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:37.277974 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf18164f-4621-47bd-acdc-fb1706c2b2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:37.278660 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf18164f-4621-47bd-acdc-fb1706c2b2fc admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:37.278953 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf18164f-4621-47bd-acdc-fb1706c2b2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:37.279230 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf18164f-4621-47bd-acdc-fb1706c2b2fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:37.279955 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf18164f-4621-47bd-acdc-fb1706c2b2fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:35:37.280266 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf18164f-4621-47bd-acdc-fb1706c2b2fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:37.282470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf18164f-4621-47bd-acdc-fb1706c2b2fc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:37.293849 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1177/2346] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:35:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:35:37.299125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61783247-0794-4c24-9f05-57849864e40b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:37.308567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61783247-0794-4c24-9f05-57849864e40b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:37.313881 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e6ada-30de-4612-833c-c23d3d620c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:37.314564 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e6ada-30de-4612-833c-c23d3d620c8b admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:37.314851 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e6ada-30de-4612-833c-c23d3d620c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:37.315132 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e6ada-30de-4612-833c-c23d3d620c8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:37.315828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e6ada-30de-4612-833c-c23d3d620c8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:37.317729 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e6ada-30de-4612-833c-c23d3d620c8b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:37.346388 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1170/2347] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:35:37 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:35:37.352724 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf18164f-4621-47bd-acdc-fb1706c2b2fc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:37.372458 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf18164f-4621-47bd-acdc-fb1706c2b2fc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:37.401317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:37.451731 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29483c9d-bf08-40fc-80de-7d369c7435c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:37.452412 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29483c9d-bf08-40fc-80de-7d369c7435c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:37.452700 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29483c9d-bf08-40fc-80de-7d369c7435c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:37.452980 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29483c9d-bf08-40fc-80de-7d369c7435c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:37.454748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29483c9d-bf08-40fc-80de-7d369c7435c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:37.456736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29483c9d-bf08-40fc-80de-7d369c7435c1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:37.490103 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1178/2348] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:35:37 2022] POST /identity/v3/users => generated 284 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:35:37.493831 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e6ada-30de-4612-833c-c23d3d620c8b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:37.521801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e6ada-30de-4612-833c-c23d3d620c8b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:37.536087 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a56f928-3a51-4419-8ede-77d3915c93f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:37.536799 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56f928-3a51-4419-8ede-77d3915c93f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:37.537075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56f928-3a51-4419-8ede-77d3915c93f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:37.537380 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a56f928-3a51-4419-8ede-77d3915c93f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:37.538343 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a56f928-3a51-4419-8ede-77d3915c93f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:37.540133 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a56f928-3a51-4419-8ede-77d3915c93f0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:37.569515 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1171/2349] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:35:37.574329 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29483c9d-bf08-40fc-80de-7d369c7435c1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:37.586309 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29483c9d-bf08-40fc-80de-7d369c7435c1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:37.591413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1fb76c-92ed-41f7-93a4-98749acca0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:37.592193 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1fb76c-92ed-41f7-93a4-98749acca0ff admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:37.592511 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1fb76c-92ed-41f7-93a4-98749acca0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:37.594796 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1fb76c-92ed-41f7-93a4-98749acca0ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:37.594796 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1fb76c-92ed-41f7-93a4-98749acca0ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:37.595505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1fb76c-92ed-41f7-93a4-98749acca0ff admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:37.606042 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1179/2350] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:35:37.610977 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a56f928-3a51-4419-8ede-77d3915c93f0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:37.621310 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a56f928-3a51-4419-8ede-77d3915c93f0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:37.627334 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92dea18-0d71-433a-8290-7674eb13c49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:37.628414 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dea18-0d71-433a-8290-7674eb13c49a admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:37.629413 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dea18-0d71-433a-8290-7674eb13c49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:37.630229 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dea18-0d71-433a-8290-7674eb13c49a admin admin] PATH_INFO: `/v3/projects/6f8829462e7d476fba635aac1cedd5be/users/c461fc99b0a24798a056f4b897da4a15/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:37.657452 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:37.657452 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92dea18-0d71-433a-8290-7674eb13c49a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f8829462e7d476fba635aac1cedd5be, user_id=c461fc99b0a24798a056f4b897da4a15, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:37.658311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92dea18-0d71-433a-8290-7674eb13c49a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:37.674873 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1172/2351] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:35:37 2022] PUT /identity/v3/projects/6f8829462e7d476fba635aac1cedd5be/users/c461fc99b0a24798a056f4b897da4a15/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:35:37.684401 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe48e9b-ebf8-4d77-b4e8-b22d502db27c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:37.684703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe48e9b-ebf8-4d77-b4e8-b22d502db27c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:37.684990 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe48e9b-ebf8-4d77-b4e8-b22d502db27c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:37.716283 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cbe48e9b-ebf8-4d77-b4e8-b22d502db27c None None] MFA Rules not processed for user `c461fc99b0a24798a056f4b897da4a15`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:37.717498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cbe48e9b-ebf8-4d77-b4e8-b22d502db27c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:37.789740 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1180/2352] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:35:37 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:35:37.802772 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92dea18-0d71-433a-8290-7674eb13c49a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:37.805280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e92dea18-0d71-433a-8290-7674eb13c49a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:37.874308 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92dea18-0d71-433a-8290-7674eb13c49a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:37.878625 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa26d25a-6a03-4d53-9a4f-c3a80eeba74e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '096234e9f15541c0b187dc8dd527b0fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:37.879326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26d25a-6a03-4d53-9a4f-c3a80eeba74e service glance] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:37.879631 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26d25a-6a03-4d53-9a4f-c3a80eeba74e service glance] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:37.879908 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa26d25a-6a03-4d53-9a4f-c3a80eeba74e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:37.888791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa26d25a-6a03-4d53-9a4f-c3a80eeba74e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:37.890275 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa26d25a-6a03-4d53-9a4f-c3a80eeba74e service glance] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:37.899707 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1173/2353] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:35:37 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:35:43.089344 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a24de-bdd3-4c62-a77a-50a07fcb3254 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:43.089875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a24de-bdd3-4c62-a77a-50a07fcb3254 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:43.089875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a24de-bdd3-4c62-a77a-50a07fcb3254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:43.107317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8c3a24de-bdd3-4c62-a77a-50a07fcb3254 None None] MFA Rules not processed for user `350a57ee307a4419974dcecf37e466e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:43.108501 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8c3a24de-bdd3-4c62-a77a-50a07fcb3254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:43.169487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1181/2354] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:35:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:35:47.895128 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=29,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:53.161042 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:35:54.908120 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a350c-8651-462f-9e31-4d0b49315ffd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:54.908570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a350c-8651-462f-9e31-4d0b49315ffd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:54.908687 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a350c-8651-462f-9e31-4d0b49315ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:54.936054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e25a350c-8651-462f-9e31-4d0b49315ffd None None] MFA Rules not processed for user `8c10852a728844c6ae0a48e3772d95b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:54.937466 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e25a350c-8651-462f-9e31-4d0b49315ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:55.024505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1174/2355] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jun 29 12:35:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:35:55.037711 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a24de-bdd3-4c62-a77a-50a07fcb3254 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:55.039683 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8c3a24de-bdd3-4c62-a77a-50a07fcb3254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:55.175646 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3a24de-bdd3-4c62-a77a-50a07fcb3254 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:55.182939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422c9ad-c747-4862-8011-f827637f11d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209b0df45f76404da880a9ae0c637d67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:55.183606 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422c9ad-c747-4862-8011-f827637f11d1 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:55.183928 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422c9ad-c747-4862-8011-f827637f11d1 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:55.184701 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e422c9ad-c747-4862-8011-f827637f11d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:55.193703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422c9ad-c747-4862-8011-f827637f11d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:55.195456 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e422c9ad-c747-4862-8011-f827637f11d1 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:55.204115 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1182/2356] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 12:35:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 12:35:55.732782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315864fc-e0f9-4117-845c-6c50ee5a83cc None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:55.733074 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315864fc-e0f9-4117-845c-6c50ee5a83cc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:55.733357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315864fc-e0f9-4117-845c-6c50ee5a83cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:55.750724 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-315864fc-e0f9-4117-845c-6c50ee5a83cc None None] MFA Rules not processed for user `8c10852a728844c6ae0a48e3772d95b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:55.751921 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-315864fc-e0f9-4117-845c-6c50ee5a83cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:55.758929 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1175/2357] 10.222.0.52 () {58 vars in 967 bytes} [Wed Jun 29 12:35:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:35:55.771755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422c9ad-c747-4862-8011-f827637f11d1 service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:55.783183 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e422c9ad-c747-4862-8011-f827637f11d1 service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:55.788167 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a19dbd-16f1-4cd0-83d1-5baff4cd5ca2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209b0df45f76404da880a9ae0c637d67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:55.788835 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a19dbd-16f1-4cd0-83d1-5baff4cd5ca2 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:55.789122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a19dbd-16f1-4cd0-83d1-5baff4cd5ca2 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:55.789403 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a19dbd-16f1-4cd0-83d1-5baff4cd5ca2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:55.797494 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a19dbd-16f1-4cd0-83d1-5baff4cd5ca2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:55.799115 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a19dbd-16f1-4cd0-83d1-5baff4cd5ca2 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:55.807100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1183/2358] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 12:35:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 12:35:56.009309 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7628cfd-70af-418a-b503-00ddec83e254 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:56.009806 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7628cfd-70af-418a-b503-00ddec83e254 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:56.009914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7628cfd-70af-418a-b503-00ddec83e254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:56.030024 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e7628cfd-70af-418a-b503-00ddec83e254 None None] MFA Rules not processed for user `8c10852a728844c6ae0a48e3772d95b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:56.032130 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e7628cfd-70af-418a-b503-00ddec83e254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:56.039299 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1176/2359] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jun 29 12:35:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:35:56.051066 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a19dbd-16f1-4cd0-83d1-5baff4cd5ca2 service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:56.061741 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a19dbd-16f1-4cd0-83d1-5baff4cd5ca2 service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:56.067415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcdc47d-0891-4d31-8ba7-e10293e55bae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209b0df45f76404da880a9ae0c637d67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:56.068079 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdc47d-0891-4d31-8ba7-e10293e55bae service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:56.068366 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdc47d-0891-4d31-8ba7-e10293e55bae service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:56.068644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdc47d-0891-4d31-8ba7-e10293e55bae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:56.076703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcdc47d-0891-4d31-8ba7-e10293e55bae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:56.078990 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcdc47d-0891-4d31-8ba7-e10293e55bae service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:56.086989 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1184/2360] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 12:35:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 12:35:56.432272 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6749643-3ba2-4b62-8f89-9a8cd8859220 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:56.432584 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6749643-3ba2-4b62-8f89-9a8cd8859220 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:56.432858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6749643-3ba2-4b62-8f89-9a8cd8859220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:56.450532 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b6749643-3ba2-4b62-8f89-9a8cd8859220 None None] MFA Rules not processed for user `8c10852a728844c6ae0a48e3772d95b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:56.451730 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b6749643-3ba2-4b62-8f89-9a8cd8859220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:56.459472 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1177/2361] 10.222.0.52 () {58 vars in 967 bytes} [Wed Jun 29 12:35:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:35:56.471972 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcdc47d-0891-4d31-8ba7-e10293e55bae service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:56.482381 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcdc47d-0891-4d31-8ba7-e10293e55bae service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:56.488067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b76008b-b16e-46f3-a1f2-dbc04de82a04 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209b0df45f76404da880a9ae0c637d67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:56.488695 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b76008b-b16e-46f3-a1f2-dbc04de82a04 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:56.488979 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b76008b-b16e-46f3-a1f2-dbc04de82a04 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:56.489260 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b76008b-b16e-46f3-a1f2-dbc04de82a04 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:56.497154 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b76008b-b16e-46f3-a1f2-dbc04de82a04 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:56.498735 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b76008b-b16e-46f3-a1f2-dbc04de82a04 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:56.506946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1185/2362] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 12:35:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 12:35:59.882030 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a56b9-8b70-426a-b745-eb302749b85c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:59.882509 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a56b9-8b70-426a-b745-eb302749b85c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:59.882694 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a56b9-8b70-426a-b745-eb302749b85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:59.900949 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-af5a56b9-8b70-426a-b745-eb302749b85c None None] MFA Rules not processed for user `8c10852a728844c6ae0a48e3772d95b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:35:59.902212 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-af5a56b9-8b70-426a-b745-eb302749b85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:35:59.910079 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1178/2363] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jun 29 12:35:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:35:59.923260 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b76008b-b16e-46f3-a1f2-dbc04de82a04 service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:35:59.936846 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b76008b-b16e-46f3-a1f2-dbc04de82a04 service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:35:59.942568 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb7bf6-6ab8-4ac4-89ab-9d0b009cb90e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209b0df45f76404da880a9ae0c637d67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:35:59.943254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb7bf6-6ab8-4ac4-89ab-9d0b009cb90e service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:35:59.943715 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb7bf6-6ab8-4ac4-89ab-9d0b009cb90e service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:35:59.944000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb7bf6-6ab8-4ac4-89ab-9d0b009cb90e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:35:59.952552 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb7bf6-6ab8-4ac4-89ab-9d0b009cb90e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:35:59.954156 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb7bf6-6ab8-4ac4-89ab-9d0b009cb90e service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:35:59.962769 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1186/2364] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 12:35:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 12:36:00.736902 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a56b9-8b70-426a-b745-eb302749b85c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:36:00.738301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-af5a56b9-8b70-426a-b745-eb302749b85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:36:00.852234 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a56b9-8b70-426a-b745-eb302749b85c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:36:00.857991 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d95ea-4747-49b1-b093-106f3daa74e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:36:00.859138 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d95ea-4747-49b1-b093-106f3daa74e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:00.859674 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d95ea-4747-49b1-b093-106f3daa74e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:00.859891 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d95ea-4747-49b1-b093-106f3daa74e3 admin admin] PATH_INFO: `/v3/users/c2518c8b1d0146dcb93fd4e875389f66` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:00.863100 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d95ea-4747-49b1-b093-106f3daa74e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2518c8b1d0146dcb93fd4e875389f66)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:36:00.866306 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d95ea-4747-49b1-b093-106f3daa74e3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:36:00.914929 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:36:00.965284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-316d95ea-4747-49b1-b093-106f3daa74e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2518c8b1d0146dcb93fd4e875389f66'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:00.977247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-316d95ea-4747-49b1-b093-106f3daa74e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2518c8b1d0146dcb93fd4e875389f66'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:01.010877 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-316d95ea-4747-49b1-b093-106f3daa74e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2518c8b1d0146dcb93fd4e875389f66'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:01.012409 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-316d95ea-4747-49b1-b093-106f3daa74e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2518c8b1d0146dcb93fd4e875389f66'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:01.013428 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-316d95ea-4747-49b1-b093-106f3daa74e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2518c8b1d0146dcb93fd4e875389f66'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:01.052137 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1179/2365] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:36:00 2022] DELETE /identity/v3/users/c2518c8b1d0146dcb93fd4e875389f66 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:36:01.057444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b3d31-39bd-4da2-9a14-aeb73d0214e6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:01.057987 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b3d31-39bd-4da2-9a14-aeb73d0214e6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:01.058071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0b3d31-39bd-4da2-9a14-aeb73d0214e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:01.114000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0a0b3d31-39bd-4da2-9a14-aeb73d0214e6 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:36:01.118434 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0a0b3d31-39bd-4da2-9a14-aeb73d0214e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:36:01.248209 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:36:01.264640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1187/2366] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:36:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:36:01.273000 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d95ea-4747-49b1-b093-106f3daa74e3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:36:01.274454 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-316d95ea-4747-49b1-b093-106f3daa74e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:36:01.420012 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d95ea-4747-49b1-b093-106f3daa74e3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:36:01.424428 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2722d46-a28e-4864-b1c9-7dba0663d859 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:36:01.425090 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2722d46-a28e-4864-b1c9-7dba0663d859 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:01.425381 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2722d46-a28e-4864-b1c9-7dba0663d859 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:01.425657 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2722d46-a28e-4864-b1c9-7dba0663d859 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:01.446007 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2722d46-a28e-4864-b1c9-7dba0663d859 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:36:01.447548 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2722d46-a28e-4864-b1c9-7dba0663d859 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:36:01.462632 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1180/2367] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:36:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:36:01.700321 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b3d31-39bd-4da2-9a14-aeb73d0214e6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:36:01.701851 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0a0b3d31-39bd-4da2-9a14-aeb73d0214e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:36:01.723308 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0b3d31-39bd-4da2-9a14-aeb73d0214e6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:36:01.740963 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b831ee-1d3d-413a-a8fe-5a62505fd42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:36:01.741953 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b831ee-1d3d-413a-a8fe-5a62505fd42a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:01.742246 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b831ee-1d3d-413a-a8fe-5a62505fd42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:01.742668 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b831ee-1d3d-413a-a8fe-5a62505fd42a admin admin] PATH_INFO: `/v3/projects/f826993a33644995a3150d71cc40add7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:01.744027 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b831ee-1d3d-413a-a8fe-5a62505fd42a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f826993a33644995a3150d71cc40add7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:36:01.746041 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b831ee-1d3d-413a-a8fe-5a62505fd42a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:36:01.828919 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c7b831ee-1d3d-413a-a8fe-5a62505fd42a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f826993a33644995a3150d71cc40add7', 'request_id': 'req-c7b831ee-1d3d-413a-a8fe-5a62505fd42a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:01.841423 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c7b831ee-1d3d-413a-a8fe-5a62505fd42a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f826993a33644995a3150d71cc40add7', 'request_id': 'req-c7b831ee-1d3d-413a-a8fe-5a62505fd42a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:01.854147 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c7b831ee-1d3d-413a-a8fe-5a62505fd42a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f826993a33644995a3150d71cc40add7', 'request_id': 'req-c7b831ee-1d3d-413a-a8fe-5a62505fd42a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:01.875539 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c7b831ee-1d3d-413a-a8fe-5a62505fd42a admin admin] The token cache is being invalidate because project f826993a33644995a3150d71cc40add7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:36:01.875912 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c7b831ee-1d3d-413a-a8fe-5a62505fd42a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:01.876526 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c7b831ee-1d3d-413a-a8fe-5a62505fd42a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:01.882230 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1188/2368] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:36:01 2022] DELETE /identity/v3/projects/f826993a33644995a3150d71cc40add7 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:36:10.786154 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2722d46-a28e-4864-b1c9-7dba0663d859 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:36:10.795023 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2722d46-a28e-4864-b1c9-7dba0663d859 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:36:10.853180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ce14b-bcbe-44e5-9666-fe7193eb9885 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:36:10.853827 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ce14b-bcbe-44e5-9666-fe7193eb9885 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:10.854137 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ce14b-bcbe-44e5-9666-fe7193eb9885 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:10.854441 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ce14b-bcbe-44e5-9666-fe7193eb9885 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:10.856222 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4d2ce14b-bcbe-44e5-9666-fe7193eb9885 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:36:10.899570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:36:10.931014 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ce14b-bcbe-44e5-9666-fe7193eb9885 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:36:10.932777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ce14b-bcbe-44e5-9666-fe7193eb9885 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:36:10.945191 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1181/2369] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:36:10 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:36:11.866357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:36:13.238586 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b831ee-1d3d-413a-a8fe-5a62505fd42a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:36:13.241402 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c7b831ee-1d3d-413a-a8fe-5a62505fd42a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:36:13.318229 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b831ee-1d3d-413a-a8fe-5a62505fd42a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:36:13.323079 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be9906-c933-42dd-bd17-098d8103f349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:36:13.324024 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be9906-c933-42dd-bd17-098d8103f349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:13.324316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be9906-c933-42dd-bd17-098d8103f349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:13.324659 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be9906-c933-42dd-bd17-098d8103f349 admin admin] PATH_INFO: `/v3/users/c461fc99b0a24798a056f4b897da4a15` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:13.326128 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be9906-c933-42dd-bd17-098d8103f349 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c461fc99b0a24798a056f4b897da4a15)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:36:13.330112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be9906-c933-42dd-bd17-098d8103f349 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:36:13.387838 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b8be9906-c933-42dd-bd17-098d8103f349 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c461fc99b0a24798a056f4b897da4a15'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:13.388533 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b8be9906-c933-42dd-bd17-098d8103f349 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c461fc99b0a24798a056f4b897da4a15'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:13.404881 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b8be9906-c933-42dd-bd17-098d8103f349 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c461fc99b0a24798a056f4b897da4a15'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:13.406333 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b8be9906-c933-42dd-bd17-098d8103f349 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c461fc99b0a24798a056f4b897da4a15'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:13.417452 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b8be9906-c933-42dd-bd17-098d8103f349 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c461fc99b0a24798a056f4b897da4a15'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:13.433575 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1189/2370] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:36:13 2022] DELETE /identity/v3/users/c461fc99b0a24798a056f4b897da4a15 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:36:13.439210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb519af-5974-4034-9208-f2c366b010e0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:13.439733 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb519af-5974-4034-9208-f2c366b010e0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:13.439938 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb519af-5974-4034-9208-f2c366b010e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:13.468880 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bfb519af-5974-4034-9208-f2c366b010e0 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:36:13.471189 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bfb519af-5974-4034-9208-f2c366b010e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:36:13.562997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1182/2371] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:36:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:36:13.571936 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be9906-c933-42dd-bd17-098d8103f349 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:36:13.573263 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b8be9906-c933-42dd-bd17-098d8103f349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:36:13.650526 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be9906-c933-42dd-bd17-098d8103f349 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:36:13.654405 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c52e5b-0504-4a49-9917-37721e695988 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:36:13.655211 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c52e5b-0504-4a49-9917-37721e695988 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:13.656298 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c52e5b-0504-4a49-9917-37721e695988 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:13.656710 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c52e5b-0504-4a49-9917-37721e695988 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:13.667423 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c52e5b-0504-4a49-9917-37721e695988 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:36:13.668879 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c52e5b-0504-4a49-9917-37721e695988 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:36:13.681246 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1190/2372] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:36:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:36:14.052965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb519af-5974-4034-9208-f2c366b010e0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:36:14.055056 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bfb519af-5974-4034-9208-f2c366b010e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:36:14.079094 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb519af-5974-4034-9208-f2c366b010e0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:36:14.086326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0013f241-00eb-496a-8bd3-c1a89b4dd8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:36:14.087481 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013f241-00eb-496a-8bd3-c1a89b4dd8d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:14.087873 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013f241-00eb-496a-8bd3-c1a89b4dd8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:14.088290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013f241-00eb-496a-8bd3-c1a89b4dd8d2 admin admin] PATH_INFO: `/v3/projects/6f8829462e7d476fba635aac1cedd5be` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:14.089582 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0013f241-00eb-496a-8bd3-c1a89b4dd8d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f8829462e7d476fba635aac1cedd5be)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:36:14.092452 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0013f241-00eb-496a-8bd3-c1a89b4dd8d2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:36:14.169349 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0013f241-00eb-496a-8bd3-c1a89b4dd8d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f8829462e7d476fba635aac1cedd5be', 'request_id': 'req-0013f241-00eb-496a-8bd3-c1a89b4dd8d2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:14.177201 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0013f241-00eb-496a-8bd3-c1a89b4dd8d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f8829462e7d476fba635aac1cedd5be', 'request_id': 'req-0013f241-00eb-496a-8bd3-c1a89b4dd8d2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:14.184980 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0013f241-00eb-496a-8bd3-c1a89b4dd8d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f8829462e7d476fba635aac1cedd5be', 'request_id': 'req-0013f241-00eb-496a-8bd3-c1a89b4dd8d2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:14.200357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0013f241-00eb-496a-8bd3-c1a89b4dd8d2 admin admin] The token cache is being invalidate because project 6f8829462e7d476fba635aac1cedd5be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:36:14.200692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0013f241-00eb-496a-8bd3-c1a89b4dd8d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:14.201448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0013f241-00eb-496a-8bd3-c1a89b4dd8d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:14.203756 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1183/2373] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:36:14 2022] DELETE /identity/v3/projects/6f8829462e7d476fba635aac1cedd5be => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:36:14.208797 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c679c781-f9d5-4ff9-a6bd-bbfa9d4a923e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:14.209362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c679c781-f9d5-4ff9-a6bd-bbfa9d4a923e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:14.209649 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c679c781-f9d5-4ff9-a6bd-bbfa9d4a923e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:14.238488 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:36:14.263991 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c679c781-f9d5-4ff9-a6bd-bbfa9d4a923e None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:36:14.265764 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c679c781-f9d5-4ff9-a6bd-bbfa9d4a923e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:36:14.342616 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1191/2374] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:36:14.345539 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210ffb7-8424-4d02-aeb7-c81a022ac3a2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:14.346242 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210ffb7-8424-4d02-aeb7-c81a022ac3a2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:14.346726 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210ffb7-8424-4d02-aeb7-c81a022ac3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:14.371294 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0210ffb7-8424-4d02-aeb7-c81a022ac3a2 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:36:14.372557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0210ffb7-8424-4d02-aeb7-c81a022ac3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:36:14.381623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1184/2375] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:36:14.386131 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c679c781-f9d5-4ff9-a6bd-bbfa9d4a923e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:36:14.398902 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c679c781-f9d5-4ff9-a6bd-bbfa9d4a923e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:36:14.406662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22dbe0-23d4-41e7-9d1b-949bd7942764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:36:14.407302 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22dbe0-23d4-41e7-9d1b-949bd7942764 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:14.407590 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22dbe0-23d4-41e7-9d1b-949bd7942764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:14.407954 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22dbe0-23d4-41e7-9d1b-949bd7942764 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:14.408598 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22dbe0-23d4-41e7-9d1b-949bd7942764 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:36:14.408910 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22dbe0-23d4-41e7-9d1b-949bd7942764 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:36:14.410561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22dbe0-23d4-41e7-9d1b-949bd7942764 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:36:14.428929 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1192/2376] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:36:14.430359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210ffb7-8424-4d02-aeb7-c81a022ac3a2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:36:14.440159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210ffb7-8424-4d02-aeb7-c81a022ac3a2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:36:14.446710 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ffab9f-1a70-48a7-a151-70a8989e6e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:36:14.447384 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ffab9f-1a70-48a7-a151-70a8989e6e4f admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:14.447809 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ffab9f-1a70-48a7-a151-70a8989e6e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:14.448238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ffab9f-1a70-48a7-a151-70a8989e6e4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:14.448951 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ffab9f-1a70-48a7-a151-70a8989e6e4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:36:14.458602 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ffab9f-1a70-48a7-a151-70a8989e6e4f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:36:14.493343 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1185/2377] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:36:14 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:36:14.494591 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22dbe0-23d4-41e7-9d1b-949bd7942764 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:36:14.506751 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22dbe0-23d4-41e7-9d1b-949bd7942764 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:36:14.560097 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f70eaf-cca8-4453-92d1-919da5fec37c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:14.560440 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f70eaf-cca8-4453-92d1-919da5fec37c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:14.560782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f70eaf-cca8-4453-92d1-919da5fec37c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:14.563370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1186/2378] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jun 29 12:36:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:36:14.569485 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a69998-fe46-4638-b627-ce6a7348140c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:14.569782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a69998-fe46-4638-b627-ce6a7348140c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:14.570073 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a69998-fe46-4638-b627-ce6a7348140c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:14.611873 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab3cf8-da8d-40a6-a85a-e3a9f1ea1dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:36:14.613291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab3cf8-da8d-40a6-a85a-e3a9f1ea1dec admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:14.613291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab3cf8-da8d-40a6-a85a-e3a9f1ea1dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:14.613511 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab3cf8-da8d-40a6-a85a-e3a9f1ea1dec admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:14.614143 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab3cf8-da8d-40a6-a85a-e3a9f1ea1dec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:36:14.616667 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab3cf8-da8d-40a6-a85a-e3a9f1ea1dec admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:36:14.619367 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:36:14.645921 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1193/2379] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:36:14 2022] POST /identity/v3/users => generated 287 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:36:14.649589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab3cf8-da8d-40a6-a85a-e3a9f1ea1dec admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:36:14.650984 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-40a69998-fe46-4638-b627-ce6a7348140c None None] MFA Rules not processed for user `3ac7391a3c41499d94e5f97478797313`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:36:14.652194 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-40a69998-fe46-4638-b627-ce6a7348140c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:36:14.660264 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab3cf8-da8d-40a6-a85a-e3a9f1ea1dec admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:36:14.665270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2284cb59-7732-4496-9817-ef9db593efa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:36:14.666009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284cb59-7732-4496-9817-ef9db593efa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:14.666365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284cb59-7732-4496-9817-ef9db593efa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:14.666747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284cb59-7732-4496-9817-ef9db593efa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:14.667518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2284cb59-7732-4496-9817-ef9db593efa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:36:14.669115 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2284cb59-7732-4496-9817-ef9db593efa0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:36:14.684323 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1194/2380] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:36:14.685425 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2284cb59-7732-4496-9817-ef9db593efa0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:36:14.699313 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2284cb59-7732-4496-9817-ef9db593efa0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:36:14.708392 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a2b8ef-3871-4acc-90b5-5755d1ce65b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:36:14.709122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2b8ef-3871-4acc-90b5-5755d1ce65b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:14.709737 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2b8ef-3871-4acc-90b5-5755d1ce65b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:14.709866 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a2b8ef-3871-4acc-90b5-5755d1ce65b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:14.710955 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a2b8ef-3871-4acc-90b5-5755d1ce65b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:36:14.713004 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a2b8ef-3871-4acc-90b5-5755d1ce65b9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:36:14.726721 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1195/2381] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:36:14.730745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a2b8ef-3871-4acc-90b5-5755d1ce65b9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:36:14.740881 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a2b8ef-3871-4acc-90b5-5755d1ce65b9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:36:14.745508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3efd2f-df01-4818-a12c-fef6f201819c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:36:14.746385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3efd2f-df01-4818-a12c-fef6f201819c admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:14.746756 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3efd2f-df01-4818-a12c-fef6f201819c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:14.747041 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3efd2f-df01-4818-a12c-fef6f201819c admin admin] PATH_INFO: `/v3/projects/334ad3e0b90c47fba2f75c54ce6ef0fe/users/17fce8de2bf9436d8f5da0ce924483e5/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:14.760633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1187/2382] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jun 29 12:36:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:36:14.763591 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3efd2f-df01-4818-a12c-fef6f201819c admin admin] RBAC: Authorizing `identity:create_grant(project_id=334ad3e0b90c47fba2f75c54ce6ef0fe, user_id=17fce8de2bf9436d8f5da0ce924483e5, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:36:14.766785 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3efd2f-df01-4818-a12c-fef6f201819c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:36:14.772885 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a69998-fe46-4638-b627-ce6a7348140c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:36:14.774306 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-40a69998-fe46-4638-b627-ce6a7348140c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:36:14.781127 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1196/2383] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:36:14 2022] PUT /identity/v3/projects/334ad3e0b90c47fba2f75c54ce6ef0fe/users/17fce8de2bf9436d8f5da0ce924483e5/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:36:14.791793 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7580879-5512-4206-978d-8d9829965c13 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:14.792205 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7580879-5512-4206-978d-8d9829965c13 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:14.792592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7580879-5512-4206-978d-8d9829965c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:14.821959 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d7580879-5512-4206-978d-8d9829965c13 None None] MFA Rules not processed for user `17fce8de2bf9436d8f5da0ce924483e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:36:14.823368 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d7580879-5512-4206-978d-8d9829965c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:36:14.833494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a69998-fe46-4638-b627-ce6a7348140c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:36:14.839730 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:36:14.885555 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7baa2f0-a90a-4c12-bc83-429ce17dfbbc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '096234e9f15541c0b187dc8dd527b0fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:36:14.886122 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7baa2f0-a90a-4c12-bc83-429ce17dfbbc service glance] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:14.886407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7baa2f0-a90a-4c12-bc83-429ce17dfbbc service glance] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:14.886849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7baa2f0-a90a-4c12-bc83-429ce17dfbbc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:14.891256 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1197/2384] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:36:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:36:14.908070 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7580879-5512-4206-978d-8d9829965c13 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:36:14.909506 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d7580879-5512-4206-978d-8d9829965c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:36:14.936037 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:36:14.961587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7baa2f0-a90a-4c12-bc83-429ce17dfbbc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:36:14.963159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7baa2f0-a90a-4c12-bc83-429ce17dfbbc service glance] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:36:14.966670 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7580879-5512-4206-978d-8d9829965c13 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:36:14.971017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2aedd1-8f52-4ad5-987e-683ee0cb9814 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:36:14.971694 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2aedd1-8f52-4ad5-987e-683ee0cb9814 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:14.971974 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2aedd1-8f52-4ad5-987e-683ee0cb9814 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:14.972255 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2aedd1-8f52-4ad5-987e-683ee0cb9814 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:14.974696 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1188/2385] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:36:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 12:36:14.981029 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2aedd1-8f52-4ad5-987e-683ee0cb9814 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:36:14.982486 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2aedd1-8f52-4ad5-987e-683ee0cb9814 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:36:14.992498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1198/2386] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:36:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:36:15.044315 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9914af-ef52-4b01-bdab-efac5bb08131 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:15.044706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9914af-ef52-4b01-bdab-efac5bb08131 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:15.045065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9914af-ef52-4b01-bdab-efac5bb08131 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:15.047300 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1189/2387] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jun 29 12:36:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:36:15.052317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30306cea-3d59-4f9d-8e5c-079cc9e39dad None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:15.052615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30306cea-3d59-4f9d-8e5c-079cc9e39dad None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:15.052902 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30306cea-3d59-4f9d-8e5c-079cc9e39dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:15.074941 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-30306cea-3d59-4f9d-8e5c-079cc9e39dad None None] MFA Rules not processed for user `ce922cdbf2d34f10b6e49631f0c6d770`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:36:15.076234 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-30306cea-3d59-4f9d-8e5c-079cc9e39dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:36:15.140437 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1199/2388] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jun 29 12:36:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:36:15.153326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9914af-ef52-4b01-bdab-efac5bb08131 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:36:15.155449 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0d9914af-ef52-4b01-bdab-efac5bb08131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:36:15.168921 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9914af-ef52-4b01-bdab-efac5bb08131 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:36:15.174224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa511ab-bb0c-45e8-9916-36693b14de47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:36:15.174960 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa511ab-bb0c-45e8-9916-36693b14de47 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:15.175304 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa511ab-bb0c-45e8-9916-36693b14de47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:15.175580 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa511ab-bb0c-45e8-9916-36693b14de47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:15.184931 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa511ab-bb0c-45e8-9916-36693b14de47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:36:15.186444 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa511ab-bb0c-45e8-9916-36693b14de47 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:36:15.196983 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1190/2389] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:36:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:36:15.304109 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9006b5d-5009-472b-be52-4cd4a3ddcea6 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:15.304641 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9006b5d-5009-472b-be52-4cd4a3ddcea6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:15.304727 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9006b5d-5009-472b-be52-4cd4a3ddcea6 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:15.306029 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1200/2390] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jun 29 12:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 29 12:36:15.312329 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b3fd4-0b4d-4826-94d7-f6258711511e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:15.312618 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b3fd4-0b4d-4826-94d7-f6258711511e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:15.312900 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880b3fd4-0b4d-4826-94d7-f6258711511e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:15.331630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-880b3fd4-0b4d-4826-94d7-f6258711511e None None] MFA Rules not processed for user `3ac7391a3c41499d94e5f97478797313`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:36:15.332975 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-880b3fd4-0b4d-4826-94d7-f6258711511e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:36:15.344008 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1191/2391] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jun 29 12:36:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:36:15.356872 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9006b5d-5009-472b-be52-4cd4a3ddcea6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:36:15.359263 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f9006b5d-5009-472b-be52-4cd4a3ddcea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:36:15.377872 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9006b5d-5009-472b-be52-4cd4a3ddcea6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:36:15.381637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1319e8-c04f-4f0a-9f34-eec3bac115ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:36:15.382744 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1319e8-c04f-4f0a-9f34-eec3bac115ef service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:15.383101 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1319e8-c04f-4f0a-9f34-eec3bac115ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:15.383482 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1319e8-c04f-4f0a-9f34-eec3bac115ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:15.395104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1319e8-c04f-4f0a-9f34-eec3bac115ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:36:15.396927 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1319e8-c04f-4f0a-9f34-eec3bac115ef service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:36:15.413024 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1201/2392] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:36:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:36:17.808555 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92517977-a530-4b6f-96ca-7310d0d18d10 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:17.809030 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92517977-a530-4b6f-96ca-7310d0d18d10 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:17.809506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92517977-a530-4b6f-96ca-7310d0d18d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:17.829044 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-92517977-a530-4b6f-96ca-7310d0d18d10 None None] MFA Rules not processed for user `8c10852a728844c6ae0a48e3772d95b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:36:17.830233 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-92517977-a530-4b6f-96ca-7310d0d18d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:36:17.885608 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1192/2393] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jun 29 12:36:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:36:17.899294 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1319e8-c04f-4f0a-9f34-eec3bac115ef service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:36:17.901057 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ae1319e8-c04f-4f0a-9f34-eec3bac115ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:36:17.957016 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:36:17.972638 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1319e8-c04f-4f0a-9f34-eec3bac115ef service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:36:17.977755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea5f9d-4908-48be-8222-5bdbe692d0eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209b0df45f76404da880a9ae0c637d67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:36:17.978416 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea5f9d-4908-48be-8222-5bdbe692d0eb service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:17.978875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea5f9d-4908-48be-8222-5bdbe692d0eb service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:17.979018 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea5f9d-4908-48be-8222-5bdbe692d0eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:17.989366 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea5f9d-4908-48be-8222-5bdbe692d0eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:36:17.990336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea5f9d-4908-48be-8222-5bdbe692d0eb service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:36:18.000042 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1202/2394] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 12:36:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 12:36:25.078281 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92517977-a530-4b6f-96ca-7310d0d18d10 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:36:25.151439 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92517977-a530-4b6f-96ca-7310d0d18d10 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:36:25.156127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b130d1-6e8a-4773-ab44-a38e154047cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:36:25.157008 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b130d1-6e8a-4773-ab44-a38e154047cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:25.157287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b130d1-6e8a-4773-ab44-a38e154047cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:25.157556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b130d1-6e8a-4773-ab44-a38e154047cc admin admin] PATH_INFO: `/v3/users/17fce8de2bf9436d8f5da0ce924483e5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:25.158791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b130d1-6e8a-4773-ab44-a38e154047cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=17fce8de2bf9436d8f5da0ce924483e5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:36:25.160731 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b130d1-6e8a-4773-ab44-a38e154047cc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:36:25.212667 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a5b130d1-6e8a-4773-ab44-a38e154047cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17fce8de2bf9436d8f5da0ce924483e5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:25.222379 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a5b130d1-6e8a-4773-ab44-a38e154047cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17fce8de2bf9436d8f5da0ce924483e5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:25.224510 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:36:25.251118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a5b130d1-6e8a-4773-ab44-a38e154047cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17fce8de2bf9436d8f5da0ce924483e5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:25.252044 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a5b130d1-6e8a-4773-ab44-a38e154047cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17fce8de2bf9436d8f5da0ce924483e5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:25.253154 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a5b130d1-6e8a-4773-ab44-a38e154047cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17fce8de2bf9436d8f5da0ce924483e5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:25.266645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1193/2395] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:36:25 2022] DELETE /identity/v3/users/17fce8de2bf9436d8f5da0ce924483e5 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:36:25.271388 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac183e3-9fa2-4c1c-a62b-cbddba832571 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:25.271920 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac183e3-9fa2-4c1c-a62b-cbddba832571 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:25.272026 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac183e3-9fa2-4c1c-a62b-cbddba832571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:25.291854 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9ac183e3-9fa2-4c1c-a62b-cbddba832571 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:36:25.293104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9ac183e3-9fa2-4c1c-a62b-cbddba832571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:36:25.361672 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1203/2396] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:36:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:36:25.371756 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b130d1-6e8a-4773-ab44-a38e154047cc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:36:25.373260 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a5b130d1-6e8a-4773-ab44-a38e154047cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:36:25.440153 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b130d1-6e8a-4773-ab44-a38e154047cc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:36:25.445187 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b408c-1dca-4eed-96bc-00eb90efebce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:36:25.446228 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b408c-1dca-4eed-96bc-00eb90efebce service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:25.446720 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b408c-1dca-4eed-96bc-00eb90efebce service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:25.447090 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b408c-1dca-4eed-96bc-00eb90efebce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:25.458173 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1b408c-1dca-4eed-96bc-00eb90efebce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:36:25.459683 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1b408c-1dca-4eed-96bc-00eb90efebce service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:36:25.470336 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1194/2397] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:36:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:36:25.831592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac183e3-9fa2-4c1c-a62b-cbddba832571 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:36:25.833126 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9ac183e3-9fa2-4c1c-a62b-cbddba832571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:36:25.845768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac183e3-9fa2-4c1c-a62b-cbddba832571 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:36:25.864539 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7bfaa-99c9-4f40-9020-e365e1b7c280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:36:25.865327 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7bfaa-99c9-4f40-9020-e365e1b7c280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:25.865611 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7bfaa-99c9-4f40-9020-e365e1b7c280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:25.865890 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7bfaa-99c9-4f40-9020-e365e1b7c280 admin admin] PATH_INFO: `/v3/projects/334ad3e0b90c47fba2f75c54ce6ef0fe` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:25.868235 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e7bfaa-99c9-4f40-9020-e365e1b7c280 admin admin] RBAC: Authorizing `identity:delete_project(project_id=334ad3e0b90c47fba2f75c54ce6ef0fe)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:36:25.870339 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e7bfaa-99c9-4f40-9020-e365e1b7c280 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:36:25.930239 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b4e7bfaa-99c9-4f40-9020-e365e1b7c280 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '334ad3e0b90c47fba2f75c54ce6ef0fe', 'request_id': 'req-b4e7bfaa-99c9-4f40-9020-e365e1b7c280'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:25.939881 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b4e7bfaa-99c9-4f40-9020-e365e1b7c280 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '334ad3e0b90c47fba2f75c54ce6ef0fe', 'request_id': 'req-b4e7bfaa-99c9-4f40-9020-e365e1b7c280'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:25.948149 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b4e7bfaa-99c9-4f40-9020-e365e1b7c280 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '334ad3e0b90c47fba2f75c54ce6ef0fe', 'request_id': 'req-b4e7bfaa-99c9-4f40-9020-e365e1b7c280'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:25.960694 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b4e7bfaa-99c9-4f40-9020-e365e1b7c280 admin admin] The token cache is being invalidate because project 334ad3e0b90c47fba2f75c54ce6ef0fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:36:25.960991 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b4e7bfaa-99c9-4f40-9020-e365e1b7c280 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:25.961646 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b4e7bfaa-99c9-4f40-9020-e365e1b7c280 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:36:25.963274 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1204/2398] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:36:25 2022] DELETE /identity/v3/projects/334ad3e0b90c47fba2f75c54ce6ef0fe => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:36:30.556849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b408c-1dca-4eed-96bc-00eb90efebce service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:36:30.558221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cb1b408c-1dca-4eed-96bc-00eb90efebce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:36:30.653965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b408c-1dca-4eed-96bc-00eb90efebce service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:36:30.707029 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:36:30.715161 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2159afd-a0cc-4117-90dd-dcce6255ff82 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98135cacb9d4f22846ad33500b9bd1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:36:30.715810 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2159afd-a0cc-4117-90dd-dcce6255ff82 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:30.716096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2159afd-a0cc-4117-90dd-dcce6255ff82 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:30.716377 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2159afd-a0cc-4117-90dd-dcce6255ff82 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:30.718734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d2159afd-a0cc-4117-90dd-dcce6255ff82 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:36:30.729788 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2159afd-a0cc-4117-90dd-dcce6255ff82 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:36:30.731397 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2159afd-a0cc-4117-90dd-dcce6255ff82 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:36:30.741823 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1195/2399] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:36:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:36:35.957522 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:36:40.737628 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:36:41.532708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e826d-14cf-4154-9a10-965b10e87354 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:41.533246 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e826d-14cf-4154-9a10-965b10e87354 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:41.533351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e826d-14cf-4154-9a10-965b10e87354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:41.586029 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-484e826d-14cf-4154-9a10-965b10e87354 None None] MFA Rules not processed for user `8c10852a728844c6ae0a48e3772d95b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:36:41.587410 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-484e826d-14cf-4154-9a10-965b10e87354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:36:41.651894 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1205/2400] 10.222.0.52 () {58 vars in 967 bytes} [Wed Jun 29 12:36:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:36:41.664591 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2159afd-a0cc-4117-90dd-dcce6255ff82 service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:36:41.666019 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d2159afd-a0cc-4117-90dd-dcce6255ff82 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:36:41.758800 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2159afd-a0cc-4117-90dd-dcce6255ff82 service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:36:41.763889 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7dec7e-e768-4063-a9d7-5e9228878c48 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209b0df45f76404da880a9ae0c637d67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:36:41.764714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7dec7e-e768-4063-a9d7-5e9228878c48 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:36:41.765006 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7dec7e-e768-4063-a9d7-5e9228878c48 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:36:41.765298 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7dec7e-e768-4063-a9d7-5e9228878c48 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:36:41.773732 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7dec7e-e768-4063-a9d7-5e9228878c48 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:36:41.775362 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7dec7e-e768-4063-a9d7-5e9228878c48 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:36:41.784358 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1196/2401] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 12:36:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 12:36:51.648946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=40 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:36:51.780765 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=41 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:37:13.858871 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b05b3-517f-4226-ae3c-0626aaab0339 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:37:13.859420 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b05b3-517f-4226-ae3c-0626aaab0339 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:37:13.859597 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b05b3-517f-4226-ae3c-0626aaab0339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:37:13.877283 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-532b05b3-517f-4226-ae3c-0626aaab0339 None None] MFA Rules not processed for user `350a57ee307a4419974dcecf37e466e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:37:13.878510 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-532b05b3-517f-4226-ae3c-0626aaab0339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:37:13.932971 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1206/2402] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:37:13 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:37:23.926386 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=25 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:37:29.694060 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d70a0-ad9c-47e5-80db-eb392ad9e345 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:37:29.694640 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d70a0-ad9c-47e5-80db-eb392ad9e345 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:37:29.694640 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4d70a0-ad9c-47e5-80db-eb392ad9e345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:37:29.711895 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4b4d70a0-ad9c-47e5-80db-eb392ad9e345 None None] MFA Rules not processed for user `6252d0519ed9421e89710ca5294524c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:37:29.713075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4b4d70a0-ad9c-47e5-80db-eb392ad9e345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:37:29.766405 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1197/2403] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:37:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:37:39.760198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=25 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:37:40.325149 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ddd7b-6a22-412f-afd6-c355349f9fe5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:37:40.325657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ddd7b-6a22-412f-afd6-c355349f9fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:37:40.325753 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91ddd7b-6a22-412f-afd6-c355349f9fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:37:40.343476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c91ddd7b-6a22-412f-afd6-c355349f9fe5 None None] MFA Rules not processed for user `a4a30b65a91a4fd89e8b59df670f60ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:37:40.344717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c91ddd7b-6a22-412f-afd6-c355349f9fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:37:40.399517 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1207/2404] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:37:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:37:50.392395 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=25 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:38:12.802237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d70a0-ad9c-47e5-80db-eb392ad9e345 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:38:12.803687 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4b4d70a0-ad9c-47e5-80db-eb392ad9e345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:38:12.869177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4d70a0-ad9c-47e5-80db-eb392ad9e345 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:38:12.872790 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b688c930-12fa-4a39-9713-35149eaae75d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:38:12.874958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688c930-12fa-4a39-9713-35149eaae75d service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:38:12.875247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688c930-12fa-4a39-9713-35149eaae75d service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:38:12.875667 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688c930-12fa-4a39-9713-35149eaae75d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:38:12.883758 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b688c930-12fa-4a39-9713-35149eaae75d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:38:12.946983 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b688c930-12fa-4a39-9713-35149eaae75d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:38:12.948351 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b688c930-12fa-4a39-9713-35149eaae75d service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:38:12.958973 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1198/2405] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:38:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:38:13.847502 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ddd7b-6a22-412f-afd6-c355349f9fe5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:38:13.849114 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c91ddd7b-6a22-412f-afd6-c355349f9fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:38:13.863116 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91ddd7b-6a22-412f-afd6-c355349f9fe5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:38:13.867658 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3280b4d-33c7-4188-971d-7a9c4ff799e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:38:13.868596 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3280b4d-33c7-4188-971d-7a9c4ff799e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:38:13.868828 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3280b4d-33c7-4188-971d-7a9c4ff799e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:38:13.869105 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3280b4d-33c7-4188-971d-7a9c4ff799e0 admin admin] PATH_INFO: `/v3/users/350a57ee307a4419974dcecf37e466e5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:38:13.870310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3280b4d-33c7-4188-971d-7a9c4ff799e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=350a57ee307a4419974dcecf37e466e5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:38:13.872277 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3280b4d-33c7-4188-971d-7a9c4ff799e0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:38:13.918201 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f3280b4d-33c7-4188-971d-7a9c4ff799e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '350a57ee307a4419974dcecf37e466e5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:38:13.918869 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f3280b4d-33c7-4188-971d-7a9c4ff799e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '350a57ee307a4419974dcecf37e466e5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:38:13.931496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f3280b4d-33c7-4188-971d-7a9c4ff799e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '350a57ee307a4419974dcecf37e466e5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:38:13.932214 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f3280b4d-33c7-4188-971d-7a9c4ff799e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '350a57ee307a4419974dcecf37e466e5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:38:13.940263 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f3280b4d-33c7-4188-971d-7a9c4ff799e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '350a57ee307a4419974dcecf37e466e5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:38:13.952905 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1208/2406] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:38:13 2022] DELETE /identity/v3/users/350a57ee307a4419974dcecf37e466e5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:38:14.134266 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b688c930-12fa-4a39-9713-35149eaae75d service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:38:14.145028 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b688c930-12fa-4a39-9713-35149eaae75d service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:38:14.215682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928cbe7b-4164-404f-a29f-0fc3d1f191d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:38:14.217372 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928cbe7b-4164-404f-a29f-0fc3d1f191d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:38:14.217516 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928cbe7b-4164-404f-a29f-0fc3d1f191d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:38:14.217702 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928cbe7b-4164-404f-a29f-0fc3d1f191d1 admin admin] PATH_INFO: `/v3/projects/8be6d4a4e2344e25b7661d69a9e222c8` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:38:14.218910 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928cbe7b-4164-404f-a29f-0fc3d1f191d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8be6d4a4e2344e25b7661d69a9e222c8)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:38:14.220808 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928cbe7b-4164-404f-a29f-0fc3d1f191d1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:38:14.270181 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-928cbe7b-4164-404f-a29f-0fc3d1f191d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8be6d4a4e2344e25b7661d69a9e222c8', 'request_id': 'req-928cbe7b-4164-404f-a29f-0fc3d1f191d1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:38:14.272341 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:38:14.276913 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-928cbe7b-4164-404f-a29f-0fc3d1f191d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8be6d4a4e2344e25b7661d69a9e222c8', 'request_id': 'req-928cbe7b-4164-404f-a29f-0fc3d1f191d1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:38:14.283532 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-928cbe7b-4164-404f-a29f-0fc3d1f191d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8be6d4a4e2344e25b7661d69a9e222c8', 'request_id': 'req-928cbe7b-4164-404f-a29f-0fc3d1f191d1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:38:14.296299 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-928cbe7b-4164-404f-a29f-0fc3d1f191d1 admin admin] The token cache is being invalidate because project 8be6d4a4e2344e25b7661d69a9e222c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:38:14.296590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-928cbe7b-4164-404f-a29f-0fc3d1f191d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:38:14.297224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-928cbe7b-4164-404f-a29f-0fc3d1f191d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:38:14.298874 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1199/2407] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:38:14 2022] DELETE /identity/v3/projects/8be6d4a4e2344e25b7661d69a9e222c8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:38:14.303948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de6f6f-c25a-41c9-b61c-b5c3b23db3ef None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:38:14.304245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de6f6f-c25a-41c9-b61c-b5c3b23db3ef None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:38:14.304532 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de6f6f-c25a-41c9-b61c-b5c3b23db3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:38:14.322559 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a2de6f6f-c25a-41c9-b61c-b5c3b23db3ef None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:38:14.323758 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a2de6f6f-c25a-41c9-b61c-b5c3b23db3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:38:14.385114 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1209/2408] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:38:14.393571 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddbe9a-fdc9-47de-aea3-25f48a69d183 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:38:14.393866 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddbe9a-fdc9-47de-aea3-25f48a69d183 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:38:14.394146 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ddbe9a-fdc9-47de-aea3-25f48a69d183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:38:14.413622 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-44ddbe9a-fdc9-47de-aea3-25f48a69d183 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:38:14.414873 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-44ddbe9a-fdc9-47de-aea3-25f48a69d183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:38:14.426404 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1200/2409] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:38:14.431629 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de6f6f-c25a-41c9-b61c-b5c3b23db3ef None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:38:14.442329 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de6f6f-c25a-41c9-b61c-b5c3b23db3ef None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:38:14.446999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8857740-6411-4d3d-8b57-e2d480bce1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:38:14.447667 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8857740-6411-4d3d-8b57-e2d480bce1fe admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:38:14.447953 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8857740-6411-4d3d-8b57-e2d480bce1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:38:14.448233 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8857740-6411-4d3d-8b57-e2d480bce1fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:38:14.448971 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8857740-6411-4d3d-8b57-e2d480bce1fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:38:14.449279 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8857740-6411-4d3d-8b57-e2d480bce1fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:38:14.451571 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8857740-6411-4d3d-8b57-e2d480bce1fe admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:38:14.463634 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1210/2410] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:38:14.468776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddbe9a-fdc9-47de-aea3-25f48a69d183 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:38:14.478170 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ddbe9a-fdc9-47de-aea3-25f48a69d183 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:38:14.483360 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300bad8d-0e23-4590-baa6-7a28fd95e7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:38:14.484017 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bad8d-0e23-4590-baa6-7a28fd95e7ee admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:38:14.484303 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bad8d-0e23-4590-baa6-7a28fd95e7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:38:14.484584 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300bad8d-0e23-4590-baa6-7a28fd95e7ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:38:14.485318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300bad8d-0e23-4590-baa6-7a28fd95e7ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:38:14.487271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300bad8d-0e23-4590-baa6-7a28fd95e7ee admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:38:14.512165 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1201/2411] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:38:14 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:38:14.516870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8857740-6411-4d3d-8b57-e2d480bce1fe admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:38:14.527265 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8857740-6411-4d3d-8b57-e2d480bce1fe admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:38:14.584203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a2093-54ab-4888-a13f-9796419dc766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:38:14.584879 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a2093-54ab-4888-a13f-9796419dc766 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:38:14.585170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a2093-54ab-4888-a13f-9796419dc766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:38:14.585451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94a2093-54ab-4888-a13f-9796419dc766 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:38:14.586175 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94a2093-54ab-4888-a13f-9796419dc766 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:38:14.588167 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94a2093-54ab-4888-a13f-9796419dc766 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:38:14.595236 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:38:14.610677 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1211/2412] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:38:14 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:38:14.614767 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300bad8d-0e23-4590-baa6-7a28fd95e7ee admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:38:14.625738 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300bad8d-0e23-4590-baa6-7a28fd95e7ee admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:38:14.630983 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ca012-8ac3-4348-ab5e-30a3bddb02f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:38:14.631643 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ca012-8ac3-4348-ab5e-30a3bddb02f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:38:14.631927 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ca012-8ac3-4348-ab5e-30a3bddb02f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:38:14.632206 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ca012-8ac3-4348-ab5e-30a3bddb02f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:38:14.632930 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ca012-8ac3-4348-ab5e-30a3bddb02f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:38:14.634571 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ca012-8ac3-4348-ab5e-30a3bddb02f5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:38:14.646655 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1202/2413] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:38:14.650601 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a2093-54ab-4888-a13f-9796419dc766 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:38:14.660687 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94a2093-54ab-4888-a13f-9796419dc766 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:38:14.665699 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265431c5-9cb8-4b6b-9276-83c2451ea57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:38:14.666364 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265431c5-9cb8-4b6b-9276-83c2451ea57a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:38:14.666688 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265431c5-9cb8-4b6b-9276-83c2451ea57a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:38:14.666970 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265431c5-9cb8-4b6b-9276-83c2451ea57a admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:38:14.667699 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265431c5-9cb8-4b6b-9276-83c2451ea57a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:38:14.669976 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265431c5-9cb8-4b6b-9276-83c2451ea57a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:38:14.681022 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1212/2414] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:38:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:38:14.685664 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ca012-8ac3-4348-ab5e-30a3bddb02f5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:38:14.696796 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ca012-8ac3-4348-ab5e-30a3bddb02f5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:38:14.701514 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbcbcdf-6c7f-4177-b12e-c7a32c8abc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:38:14.702390 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbcbcdf-6c7f-4177-b12e-c7a32c8abc54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:38:14.702706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbcbcdf-6c7f-4177-b12e-c7a32c8abc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:38:14.703122 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbcbcdf-6c7f-4177-b12e-c7a32c8abc54 admin admin] PATH_INFO: `/v3/projects/f18ffa75c23e4276a8b5b4fc3821fb87/users/2af4d4a69e9d4755bff61401f4349c78/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:38:14.730102 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbcbcdf-6c7f-4177-b12e-c7a32c8abc54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f18ffa75c23e4276a8b5b4fc3821fb87, user_id=2af4d4a69e9d4755bff61401f4349c78, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:38:14.732889 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbcbcdf-6c7f-4177-b12e-c7a32c8abc54 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:38:14.748731 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1203/2415] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:38:14 2022] PUT /identity/v3/projects/f18ffa75c23e4276a8b5b4fc3821fb87/users/2af4d4a69e9d4755bff61401f4349c78/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:38:14.754826 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a596b1-ddbe-460b-bbb7-815d31e9e6b1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:38:14.755125 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a596b1-ddbe-460b-bbb7-815d31e9e6b1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:38:14.755413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a596b1-ddbe-460b-bbb7-815d31e9e6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:38:14.775505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-18a596b1-ddbe-460b-bbb7-815d31e9e6b1 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:38:14.776939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-18a596b1-ddbe-460b-bbb7-815d31e9e6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:38:14.843173 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1213/2416] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:38:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:38:14.853975 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbcbcdf-6c7f-4177-b12e-c7a32c8abc54 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:38:14.857069 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bbbcbcdf-6c7f-4177-b12e-c7a32c8abc54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:38:14.926881 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbcbcdf-6c7f-4177-b12e-c7a32c8abc54 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:38:14.931097 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2eae6e-b504-42d9-a1d1-300cc936f7cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:38:14.932440 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2eae6e-b504-42d9-a1d1-300cc936f7cb service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:38:14.933026 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2eae6e-b504-42d9-a1d1-300cc936f7cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:38:14.933775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2eae6e-b504-42d9-a1d1-300cc936f7cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:38:14.943651 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2eae6e-b504-42d9-a1d1-300cc936f7cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:38:14.945088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2eae6e-b504-42d9-a1d1-300cc936f7cb service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:38:14.955643 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1204/2417] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:38:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:38:18.666631 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2324c96-76f8-449f-8123-c74f0a20630d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:38:18.667127 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2324c96-76f8-449f-8123-c74f0a20630d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:38:18.667127 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2324c96-76f8-449f-8123-c74f0a20630d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:38:18.706212 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b2324c96-76f8-449f-8123-c74f0a20630d None None] MFA Rules not processed for user `2af4d4a69e9d4755bff61401f4349c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:38:18.707566 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b2324c96-76f8-449f-8123-c74f0a20630d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:38:18.806295 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1214/2418] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:38:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:38:18.814880 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2eae6e-b504-42d9-a1d1-300cc936f7cb service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:38:18.816363 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7e2eae6e-b504-42d9-a1d1-300cc936f7cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:38:18.907440 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2eae6e-b504-42d9-a1d1-300cc936f7cb service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:38:18.928346 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d98e5-9eb7-4df1-9875-fed16338733a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:38:18.929224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d98e5-9eb7-4df1-9875-fed16338733a service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:38:18.929511 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d98e5-9eb7-4df1-9875-fed16338733a service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:38:18.929791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d98e5-9eb7-4df1-9875-fed16338733a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:38:18.941538 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d98e5-9eb7-4df1-9875-fed16338733a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:38:18.943217 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637d98e5-9eb7-4df1-9875-fed16338733a service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:38:18.954937 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1205/2419] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:38:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:38:20.544544 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d298e9e-f36d-42b4-95b5-4dc006841101 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:38:20.546066 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d298e9e-f36d-42b4-95b5-4dc006841101 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:38:20.546066 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d298e9e-f36d-42b4-95b5-4dc006841101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:38:20.565064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1d298e9e-f36d-42b4-95b5-4dc006841101 None None] MFA Rules not processed for user `2af4d4a69e9d4755bff61401f4349c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:38:20.566306 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1d298e9e-f36d-42b4-95b5-4dc006841101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:38:20.574515 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1215/2420] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:38:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:38:20.639571 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d98e5-9eb7-4df1-9875-fed16338733a service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:38:20.640485 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d298e9e-f36d-42b4-95b5-4dc006841101 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:38:20.644470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1d298e9e-f36d-42b4-95b5-4dc006841101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:38:20.655620 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:38:20.660141 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d98e5-9eb7-4df1-9875-fed16338733a service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:38:20.667721 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f653a1-a0cc-46c7-a777-3a9e707fa7cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:38:20.669148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f653a1-a0cc-46c7-a777-3a9e707fa7cb service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:38:20.669826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f653a1-a0cc-46c7-a777-3a9e707fa7cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:38:20.670528 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f653a1-a0cc-46c7-a777-3a9e707fa7cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:38:20.672636 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-45f653a1-a0cc-46c7-a777-3a9e707fa7cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:38:20.686951 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f653a1-a0cc-46c7-a777-3a9e707fa7cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:38:20.688393 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f653a1-a0cc-46c7-a777-3a9e707fa7cb service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:38:20.702016 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1206/2421] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:38:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 12:38:20.735596 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:38:20.746768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d298e9e-f36d-42b4-95b5-4dc006841101 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:38:20.754407 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51e96e-f6b6-4002-8dea-7775379c3472 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '096234e9f15541c0b187dc8dd527b0fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:38:20.755375 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51e96e-f6b6-4002-8dea-7775379c3472 service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:38:20.755814 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51e96e-f6b6-4002-8dea-7775379c3472 service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:38:20.756234 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51e96e-f6b6-4002-8dea-7775379c3472 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:38:20.758098 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ac51e96e-f6b6-4002-8dea-7775379c3472 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:38:20.774645 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac51e96e-f6b6-4002-8dea-7775379c3472 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:38:20.776351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac51e96e-f6b6-4002-8dea-7775379c3472 service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:38:20.790122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1216/2422] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:38:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:38:30.696694 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:38:30.784535 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:38:35.361731 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f653a1-a0cc-46c7-a777-3a9e707fa7cb service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:38:35.363207 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-45f653a1-a0cc-46c7-a777-3a9e707fa7cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:38:35.440370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f653a1-a0cc-46c7-a777-3a9e707fa7cb service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:38:35.443985 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159bd9b7-c898-4917-998b-835d599e25ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:38:35.444620 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bd9b7-c898-4917-998b-835d599e25ad service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:38:35.444902 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bd9b7-c898-4917-998b-835d599e25ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:38:35.445179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bd9b7-c898-4917-998b-835d599e25ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:38:35.446943 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-159bd9b7-c898-4917-998b-835d599e25ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:38:35.506044 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159bd9b7-c898-4917-998b-835d599e25ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:38:35.507533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159bd9b7-c898-4917-998b-835d599e25ad service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:38:35.517661 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1207/2423] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:38:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:38:45.065176 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51e96e-f6b6-4002-8dea-7775379c3472 service glance] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:38:45.078521 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51e96e-f6b6-4002-8dea-7775379c3472 service glance] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:38:45.088408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba33e99-0c01-45fb-bc0e-df8b1483ce1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:38:45.089074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba33e99-0c01-45fb-bc0e-df8b1483ce1b service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:38:45.089365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba33e99-0c01-45fb-bc0e-df8b1483ce1b service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:38:45.089652 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba33e99-0c01-45fb-bc0e-df8b1483ce1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:38:45.091443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8ba33e99-0c01-45fb-bc0e-df8b1483ce1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:38:45.155529 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba33e99-0c01-45fb-bc0e-df8b1483ce1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:38:45.157155 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba33e99-0c01-45fb-bc0e-df8b1483ce1b service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:38:45.166388 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1217/2424] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:38:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:38:45.512532 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=57 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:38:50.401294 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159bd9b7-c898-4917-998b-835d599e25ad service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:38:50.403105 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-159bd9b7-c898-4917-998b-835d599e25ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:38:50.416579 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159bd9b7-c898-4917-998b-835d599e25ad service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:38:50.420208 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fdd9d0-92fc-404c-aba1-05a877f58924 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:38:50.420861 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdd9d0-92fc-404c-aba1-05a877f58924 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:38:50.421150 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdd9d0-92fc-404c-aba1-05a877f58924 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:38:50.421523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdd9d0-92fc-404c-aba1-05a877f58924 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:38:50.423376 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-36fdd9d0-92fc-404c-aba1-05a877f58924 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:38:50.435996 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fdd9d0-92fc-404c-aba1-05a877f58924 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:38:50.437523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fdd9d0-92fc-404c-aba1-05a877f58924 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:38:50.448826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1208/2425] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:38:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:38:55.162056 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=28 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:38:56.146772 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba33e99-0c01-45fb-bc0e-df8b1483ce1b service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:38:56.148364 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8ba33e99-0c01-45fb-bc0e-df8b1483ce1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:38:56.183110 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba33e99-0c01-45fb-bc0e-df8b1483ce1b service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:38:56.189886 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d162fb-c19d-4a50-b661-90436d0e4d82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:38:56.190722 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d162fb-c19d-4a50-b661-90436d0e4d82 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:38:56.191300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d162fb-c19d-4a50-b661-90436d0e4d82 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:38:56.191740 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d162fb-c19d-4a50-b661-90436d0e4d82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:38:56.195445 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-63d162fb-c19d-4a50-b661-90436d0e4d82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:38:56.209271 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d162fb-c19d-4a50-b661-90436d0e4d82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:38:56.210830 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d162fb-c19d-4a50-b661-90436d0e4d82 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:38:56.223462 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1218/2426] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:38:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:39:00.443887 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:01.193261 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fdd9d0-92fc-404c-aba1-05a877f58924 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:01.201926 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fdd9d0-92fc-404c-aba1-05a877f58924 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:01.206583 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dab4a3-4079-40b1-a49e-5ed491dbd2b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:01.207221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dab4a3-4079-40b1-a49e-5ed491dbd2b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:01.207501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dab4a3-4079-40b1-a49e-5ed491dbd2b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:01.207779 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dab4a3-4079-40b1-a49e-5ed491dbd2b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:01.210084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-13dab4a3-4079-40b1-a49e-5ed491dbd2b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:01.223180 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dab4a3-4079-40b1-a49e-5ed491dbd2b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:01.224604 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dab4a3-4079-40b1-a49e-5ed491dbd2b6 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:01.234076 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1209/2427] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:39:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:39:04.418015 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d162fb-c19d-4a50-b661-90436d0e4d82 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:04.421192 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-63d162fb-c19d-4a50-b661-90436d0e4d82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:04.478819 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dab4a3-4079-40b1-a49e-5ed491dbd2b6 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:04.480331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-13dab4a3-4079-40b1-a49e-5ed491dbd2b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:04.493581 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dab4a3-4079-40b1-a49e-5ed491dbd2b6 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:04.496584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d162fb-c19d-4a50-b661-90436d0e4d82 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:04.498350 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21034e-6c31-4229-89c0-77b0370eb1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:04.499262 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21034e-6c31-4229-89c0-77b0370eb1db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:04.499551 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21034e-6c31-4229-89c0-77b0370eb1db admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:04.499861 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21034e-6c31-4229-89c0-77b0370eb1db admin admin] PATH_INFO: `/v3/users/a4a30b65a91a4fd89e8b59df670f60ad` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:04.501139 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21034e-6c31-4229-89c0-77b0370eb1db admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4a30b65a91a4fd89e8b59df670f60ad)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:04.501652 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9a2fb-509a-4e51-bd82-d93a2560c2f1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98135cacb9d4f22846ad33500b9bd1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:04.502295 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9a2fb-509a-4e51-bd82-d93a2560c2f1 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:04.502627 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9a2fb-509a-4e51-bd82-d93a2560c2f1 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:04.502931 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9a2fb-509a-4e51-bd82-d93a2560c2f1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:04.503119 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21034e-6c31-4229-89c0-77b0370eb1db admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:04.504681 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bef9a2fb-509a-4e51-bd82-d93a2560c2f1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:04.521335 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef9a2fb-509a-4e51-bd82-d93a2560c2f1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:04.523093 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef9a2fb-509a-4e51-bd82-d93a2560c2f1 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:04.535503 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1219/2428] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:39:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:39:04.554985 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2f21034e-6c31-4229-89c0-77b0370eb1db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4a30b65a91a4fd89e8b59df670f60ad'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:04.563022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2f21034e-6c31-4229-89c0-77b0370eb1db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4a30b65a91a4fd89e8b59df670f60ad'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:04.575253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2f21034e-6c31-4229-89c0-77b0370eb1db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4a30b65a91a4fd89e8b59df670f60ad'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:04.575892 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2f21034e-6c31-4229-89c0-77b0370eb1db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4a30b65a91a4fd89e8b59df670f60ad'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:04.576489 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2f21034e-6c31-4229-89c0-77b0370eb1db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4a30b65a91a4fd89e8b59df670f60ad'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:04.588944 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1210/2429] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:39:04 2022] DELETE /identity/v3/users/a4a30b65a91a4fd89e8b59df670f60ad => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:39:04.767088 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9a2fb-509a-4e51-bd82-d93a2560c2f1 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:04.784334 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9a2fb-509a-4e51-bd82-d93a2560c2f1 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:04.842909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0642b430-a94c-4e0c-a907-622ea9f560c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:04.843730 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642b430-a94c-4e0c-a907-622ea9f560c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:04.844051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642b430-a94c-4e0c-a907-622ea9f560c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:04.844883 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642b430-a94c-4e0c-a907-622ea9f560c9 admin admin] PATH_INFO: `/v3/projects/0f625199b4eb48c98944ea79a90f2841` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:04.846115 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0642b430-a94c-4e0c-a907-622ea9f560c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f625199b4eb48c98944ea79a90f2841)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:04.848061 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0642b430-a94c-4e0c-a907-622ea9f560c9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:04.895967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0642b430-a94c-4e0c-a907-622ea9f560c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f625199b4eb48c98944ea79a90f2841', 'request_id': 'req-0642b430-a94c-4e0c-a907-622ea9f560c9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:04.903075 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0642b430-a94c-4e0c-a907-622ea9f560c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f625199b4eb48c98944ea79a90f2841', 'request_id': 'req-0642b430-a94c-4e0c-a907-622ea9f560c9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:04.909297 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0642b430-a94c-4e0c-a907-622ea9f560c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f625199b4eb48c98944ea79a90f2841', 'request_id': 'req-0642b430-a94c-4e0c-a907-622ea9f560c9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:04.919365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0642b430-a94c-4e0c-a907-622ea9f560c9 admin admin] The token cache is being invalidate because project 0f625199b4eb48c98944ea79a90f2841 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:39:04.919651 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0642b430-a94c-4e0c-a907-622ea9f560c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:04.920278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0642b430-a94c-4e0c-a907-622ea9f560c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:04.921803 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1220/2430] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:39:04 2022] DELETE /identity/v3/projects/0f625199b4eb48c98944ea79a90f2841 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:39:04.926744 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d833217b-7101-48bd-a8bf-f4b484772eba None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:04.927063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d833217b-7101-48bd-a8bf-f4b484772eba None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:04.927375 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d833217b-7101-48bd-a8bf-f4b484772eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:04.945171 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d833217b-7101-48bd-a8bf-f4b484772eba None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:39:04.946395 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d833217b-7101-48bd-a8bf-f4b484772eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:05.006867 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1211/2431] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:39:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:39:05.015580 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b59db4-3e29-4eed-912c-57b4188300c9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:05.015877 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b59db4-3e29-4eed-912c-57b4188300c9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:05.016751 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b59db4-3e29-4eed-912c-57b4188300c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:05.029635 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:05.036291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d2b59db4-3e29-4eed-912c-57b4188300c9 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:39:05.038290 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d2b59db4-3e29-4eed-912c-57b4188300c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:05.046681 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1221/2432] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:39:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:39:05.051272 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d833217b-7101-48bd-a8bf-f4b484772eba None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:05.060872 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d833217b-7101-48bd-a8bf-f4b484772eba None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:05.066192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f200da-9002-48eb-a8d8-3e1562d273ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:05.066878 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f200da-9002-48eb-a8d8-3e1562d273ef admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:05.067166 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f200da-9002-48eb-a8d8-3e1562d273ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:05.067479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f200da-9002-48eb-a8d8-3e1562d273ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:05.068206 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f200da-9002-48eb-a8d8-3e1562d273ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:39:05.068518 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f200da-9002-48eb-a8d8-3e1562d273ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:05.070127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f200da-9002-48eb-a8d8-3e1562d273ef admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:05.082359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1212/2433] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:39:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:39:05.086532 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b59db4-3e29-4eed-912c-57b4188300c9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:05.095929 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b59db4-3e29-4eed-912c-57b4188300c9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:05.100586 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468a4a0-b686-4315-b35c-1adbf7a8c627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:05.101234 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468a4a0-b686-4315-b35c-1adbf7a8c627 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:05.101553 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468a4a0-b686-4315-b35c-1adbf7a8c627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:05.101836 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b468a4a0-b686-4315-b35c-1adbf7a8c627 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:05.102590 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b468a4a0-b686-4315-b35c-1adbf7a8c627 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:05.104491 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b468a4a0-b686-4315-b35c-1adbf7a8c627 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:05.134708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1222/2434] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:39:05 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:39:05.135921 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f200da-9002-48eb-a8d8-3e1562d273ef admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:05.147071 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f200da-9002-48eb-a8d8-3e1562d273ef admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:05.191795 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:05.209945 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4051b6e-c476-4f95-a738-74b3dfdd52cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:05.210679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4051b6e-c476-4f95-a738-74b3dfdd52cb admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:05.210962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4051b6e-c476-4f95-a738-74b3dfdd52cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:05.211238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4051b6e-c476-4f95-a738-74b3dfdd52cb admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:05.211926 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4051b6e-c476-4f95-a738-74b3dfdd52cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:05.213841 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4051b6e-c476-4f95-a738-74b3dfdd52cb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:05.237250 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1213/2435] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:39:05 2022] POST /identity/v3/users => generated 282 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:39:05.240665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468a4a0-b686-4315-b35c-1adbf7a8c627 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:05.251815 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b468a4a0-b686-4315-b35c-1adbf7a8c627 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:05.256567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7bbdc-7421-441b-b77a-56d939d12ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:05.257368 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7bbdc-7421-441b-b77a-56d939d12ced admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:05.257604 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7bbdc-7421-441b-b77a-56d939d12ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:05.257938 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7bbdc-7421-441b-b77a-56d939d12ced admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:05.258713 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c7bbdc-7421-441b-b77a-56d939d12ced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:05.260313 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c7bbdc-7421-441b-b77a-56d939d12ced admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:05.275004 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1223/2436] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:39:05.276380 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4051b6e-c476-4f95-a738-74b3dfdd52cb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:05.286371 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4051b6e-c476-4f95-a738-74b3dfdd52cb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:05.291104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500f4bf-b3f9-414e-985d-ee334dd30ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:05.291840 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500f4bf-b3f9-414e-985d-ee334dd30ea6 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:05.292123 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500f4bf-b3f9-414e-985d-ee334dd30ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:05.292401 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500f4bf-b3f9-414e-985d-ee334dd30ea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:05.293114 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500f4bf-b3f9-414e-985d-ee334dd30ea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:05.294730 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500f4bf-b3f9-414e-985d-ee334dd30ea6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:05.305417 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1214/2437] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:39:05.309798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7bbdc-7421-441b-b77a-56d939d12ced admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:05.322977 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7bbdc-7421-441b-b77a-56d939d12ced admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:05.327547 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f652b-24d6-406b-90fb-9f1ea8c7a894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:05.328516 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f652b-24d6-406b-90fb-9f1ea8c7a894 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:05.328802 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f652b-24d6-406b-90fb-9f1ea8c7a894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:05.329078 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f652b-24d6-406b-90fb-9f1ea8c7a894 admin admin] PATH_INFO: `/v3/projects/522f8edfe7114a028b359c8dd9faffbe/users/302bb0646019446cbd160918a3044c96/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:05.343504 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f652b-24d6-406b-90fb-9f1ea8c7a894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=522f8edfe7114a028b359c8dd9faffbe, user_id=302bb0646019446cbd160918a3044c96, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:05.346478 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4f652b-24d6-406b-90fb-9f1ea8c7a894 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:05.363681 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1224/2438] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:39:05 2022] PUT /identity/v3/projects/522f8edfe7114a028b359c8dd9faffbe/users/302bb0646019446cbd160918a3044c96/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:39:05.373557 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b7ed7-872f-476a-91ea-8fc90da4ef78 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:05.373857 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b7ed7-872f-476a-91ea-8fc90da4ef78 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:05.374145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b7ed7-872f-476a-91ea-8fc90da4ef78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:05.401129 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c87b7ed7-872f-476a-91ea-8fc90da4ef78 None None] MFA Rules not processed for user `302bb0646019446cbd160918a3044c96`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:39:05.402365 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c87b7ed7-872f-476a-91ea-8fc90da4ef78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:05.465619 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1215/2439] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:39:05 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:39:05.470351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f652b-24d6-406b-90fb-9f1ea8c7a894 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:05.532184 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4f652b-24d6-406b-90fb-9f1ea8c7a894 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:05.536641 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd2b0f-fbfa-4207-a866-09fbb88c6f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:05.537299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd2b0f-fbfa-4207-a866-09fbb88c6f83 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:05.537591 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd2b0f-fbfa-4207-a866-09fbb88c6f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:05.537869 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dd2b0f-fbfa-4207-a866-09fbb88c6f83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:05.538602 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dd2b0f-fbfa-4207-a866-09fbb88c6f83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:05.540500 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dd2b0f-fbfa-4207-a866-09fbb88c6f83 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:05.566495 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1225/2440] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:39:05 2022] POST /identity/v3/projects => generated 365 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:39:05.570561 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b7ed7-872f-476a-91ea-8fc90da4ef78 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:05.581384 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b7ed7-872f-476a-91ea-8fc90da4ef78 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:05.636640 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84e43b-0ade-4a43-9a0a-7f19408e7090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:05.637299 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84e43b-0ade-4a43-9a0a-7f19408e7090 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:05.637580 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84e43b-0ade-4a43-9a0a-7f19408e7090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:05.637858 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84e43b-0ade-4a43-9a0a-7f19408e7090 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:05.638577 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84e43b-0ade-4a43-9a0a-7f19408e7090 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:05.640493 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84e43b-0ade-4a43-9a0a-7f19408e7090 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:05.665143 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1216/2441] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:39:05 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:39:05.669186 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd2b0f-fbfa-4207-a866-09fbb88c6f83 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:05.682812 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dd2b0f-fbfa-4207-a866-09fbb88c6f83 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:05.687870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fe13b-9505-416e-95d1-0cd3bca1b2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:05.688528 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fe13b-9505-416e-95d1-0cd3bca1b2d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:05.688812 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fe13b-9505-416e-95d1-0cd3bca1b2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:05.689087 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fe13b-9505-416e-95d1-0cd3bca1b2d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:05.689964 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22fe13b-9505-416e-95d1-0cd3bca1b2d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:05.691567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22fe13b-9505-416e-95d1-0cd3bca1b2d3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:05.702989 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1226/2442] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:39:05.707416 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84e43b-0ade-4a43-9a0a-7f19408e7090 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:05.717811 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84e43b-0ade-4a43-9a0a-7f19408e7090 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:05.724209 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2694274e-f857-4145-a2dd-998eca7a01e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:05.725108 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694274e-f857-4145-a2dd-998eca7a01e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:05.725390 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694274e-f857-4145-a2dd-998eca7a01e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:05.725663 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694274e-f857-4145-a2dd-998eca7a01e8 admin admin] PATH_INFO: `/v3/projects/2591c013916e4234a28200f427fa2294/users/939abacb103c49e0a190e1bf59ebd31f/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:05.741013 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2694274e-f857-4145-a2dd-998eca7a01e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2591c013916e4234a28200f427fa2294, user_id=939abacb103c49e0a190e1bf59ebd31f, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:05.743706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2694274e-f857-4145-a2dd-998eca7a01e8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:05.756101 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1217/2443] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:39:05 2022] PUT /identity/v3/projects/2591c013916e4234a28200f427fa2294/users/939abacb103c49e0a190e1bf59ebd31f/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:39:05.760161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fe13b-9505-416e-95d1-0cd3bca1b2d3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:05.770846 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fe13b-9505-416e-95d1-0cd3bca1b2d3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:05.827825 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc7f8a-6e36-43e9-ad74-74f613c12084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:05.828454 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc7f8a-6e36-43e9-ad74-74f613c12084 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:05.829196 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc7f8a-6e36-43e9-ad74-74f613c12084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:05.829475 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc7f8a-6e36-43e9-ad74-74f613c12084 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:05.830200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc7f8a-6e36-43e9-ad74-74f613c12084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:05.831836 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc7f8a-6e36-43e9-ad74-74f613c12084 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:05.841373 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1227/2444] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:39:05 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:39:05.846058 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2694274e-f857-4145-a2dd-998eca7a01e8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:05.856276 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2694274e-f857-4145-a2dd-998eca7a01e8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:05.861578 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa9823-ab04-477d-8339-0a57627835b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:05.862483 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa9823-ab04-477d-8339-0a57627835b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:05.862774 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa9823-ab04-477d-8339-0a57627835b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:05.863059 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa9823-ab04-477d-8339-0a57627835b0 admin admin] PATH_INFO: `/v3/projects/2591c013916e4234a28200f427fa2294/users/939abacb103c49e0a190e1bf59ebd31f/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:05.865025 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa9823-ab04-477d-8339-0a57627835b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2591c013916e4234a28200f427fa2294, user_id=939abacb103c49e0a190e1bf59ebd31f, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:05.867730 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa9823-ab04-477d-8339-0a57627835b0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:05.878431 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1218/2445] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:39:05 2022] PUT /identity/v3/projects/2591c013916e4234a28200f427fa2294/users/939abacb103c49e0a190e1bf59ebd31f/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:39:05.888410 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6cd11-f85b-4d2d-8004-3f7919492b5b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:05.888698 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6cd11-f85b-4d2d-8004-3f7919492b5b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:05.888982 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6cd11-f85b-4d2d-8004-3f7919492b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:05.917366 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:05.920396 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c6c6cd11-f85b-4d2d-8004-3f7919492b5b None None] MFA Rules not processed for user `939abacb103c49e0a190e1bf59ebd31f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:39:05.923171 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c6c6cd11-f85b-4d2d-8004-3f7919492b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:05.992564 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1228/2446] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:39:05 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:39:06.003945 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa9823-ab04-477d-8339-0a57627835b0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:06.005617 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f3fa9823-ab04-477d-8339-0a57627835b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:06.015187 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:06.063454 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa9823-ab04-477d-8339-0a57627835b0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:06.067041 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146cb74-9c96-48a9-9bc3-206f552ce21e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:06.067672 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146cb74-9c96-48a9-9bc3-206f552ce21e service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:06.067952 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146cb74-9c96-48a9-9bc3-206f552ce21e service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:06.068229 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146cb74-9c96-48a9-9bc3-206f552ce21e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:06.077050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146cb74-9c96-48a9-9bc3-206f552ce21e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:06.079096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146cb74-9c96-48a9-9bc3-206f552ce21e service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:06.088350 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1219/2447] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:39:06 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:39:06.160767 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6cd11-f85b-4d2d-8004-3f7919492b5b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:06.170318 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6cd11-f85b-4d2d-8004-3f7919492b5b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:06.174626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a65e2f-827f-40b1-88f7-3dec0dce71c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:06.175262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a65e2f-827f-40b1-88f7-3dec0dce71c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:06.175544 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a65e2f-827f-40b1-88f7-3dec0dce71c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:06.175822 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a65e2f-827f-40b1-88f7-3dec0dce71c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:06.232007 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a65e2f-827f-40b1-88f7-3dec0dce71c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:06.233460 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a65e2f-827f-40b1-88f7-3dec0dce71c5 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:06.242199 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1229/2448] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:39:06 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:39:12.946648 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59413d-78c1-4adc-aa40-b0c81d9b023d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:12.947125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59413d-78c1-4adc-aa40-b0c81d9b023d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:12.947260 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59413d-78c1-4adc-aa40-b0c81d9b023d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:12.966032 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ff59413d-78c1-4adc-aa40-b0c81d9b023d None None] MFA Rules not processed for user `8c10852a728844c6ae0a48e3772d95b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:39:12.967240 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ff59413d-78c1-4adc-aa40-b0c81d9b023d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:13.027948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1220/2449] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jun 29 12:39:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:39:13.074085 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a65e2f-827f-40b1-88f7-3dec0dce71c5 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:13.075505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e7a65e2f-827f-40b1-88f7-3dec0dce71c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:13.146067 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a65e2f-827f-40b1-88f7-3dec0dce71c5 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:13.150944 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a641bd-1481-44f7-a7b4-8336a76bfb71 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209b0df45f76404da880a9ae0c637d67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:13.151585 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a641bd-1481-44f7-a7b4-8336a76bfb71 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:13.151867 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a641bd-1481-44f7-a7b4-8336a76bfb71 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:13.152141 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a641bd-1481-44f7-a7b4-8336a76bfb71 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:13.160462 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a641bd-1481-44f7-a7b4-8336a76bfb71 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:13.162030 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a641bd-1481-44f7-a7b4-8336a76bfb71 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:13.169907 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1230/2450] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 12:39:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 12:39:14.587403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59413d-78c1-4adc-aa40-b0c81d9b023d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:14.589403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ff59413d-78c1-4adc-aa40-b0c81d9b023d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:14.725796 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59413d-78c1-4adc-aa40-b0c81d9b023d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:14.733932 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d2e4b-0698-4ba6-bd00-c818fbf157bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:14.735271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d2e4b-0698-4ba6-bd00-c818fbf157bc service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:14.735560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d2e4b-0698-4ba6-bd00-c818fbf157bc service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:14.735840 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d2e4b-0698-4ba6-bd00-c818fbf157bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:14.737727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-777d2e4b-0698-4ba6-bd00-c818fbf157bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:14.752750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d2e4b-0698-4ba6-bd00-c818fbf157bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:14.754678 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d2e4b-0698-4ba6-bd00-c818fbf157bc service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:14.767483 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1221/2451] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:39:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:39:16.680217 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a641bd-1481-44f7-a7b4-8336a76bfb71 service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:16.722836 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:16.763774 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a641bd-1481-44f7-a7b4-8336a76bfb71 service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:16.768679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4247af-f223-40e9-bfc8-3af3df95725c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:16.769556 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4247af-f223-40e9-bfc8-3af3df95725c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:16.769968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4247af-f223-40e9-bfc8-3af3df95725c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:16.770251 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4247af-f223-40e9-bfc8-3af3df95725c admin admin] PATH_INFO: `/v3/users/302bb0646019446cbd160918a3044c96` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:16.771738 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4247af-f223-40e9-bfc8-3af3df95725c admin admin] RBAC: Authorizing `identity:delete_user(user_id=302bb0646019446cbd160918a3044c96)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:16.774546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4247af-f223-40e9-bfc8-3af3df95725c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:16.835662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dc4247af-f223-40e9-bfc8-3af3df95725c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '302bb0646019446cbd160918a3044c96'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:16.836349 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dc4247af-f223-40e9-bfc8-3af3df95725c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '302bb0646019446cbd160918a3044c96'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:16.849963 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dc4247af-f223-40e9-bfc8-3af3df95725c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '302bb0646019446cbd160918a3044c96'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:16.850753 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dc4247af-f223-40e9-bfc8-3af3df95725c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '302bb0646019446cbd160918a3044c96'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:16.858157 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dc4247af-f223-40e9-bfc8-3af3df95725c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '302bb0646019446cbd160918a3044c96'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:16.873243 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1231/2452] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:39:16 2022] DELETE /identity/v3/users/302bb0646019446cbd160918a3044c96 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:39:16.878745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7505e-502d-41fb-96af-21794c97d383 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:16.879611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7505e-502d-41fb-96af-21794c97d383 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:16.879899 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7505e-502d-41fb-96af-21794c97d383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:16.899249 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-dcb7505e-502d-41fb-96af-21794c97d383 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:39:16.900492 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dcb7505e-502d-41fb-96af-21794c97d383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:16.943704 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:16.976249 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1222/2453] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:39:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:39:16.987912 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4247af-f223-40e9-bfc8-3af3df95725c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:16.989416 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dc4247af-f223-40e9-bfc8-3af3df95725c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:17.071645 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4247af-f223-40e9-bfc8-3af3df95725c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:17.075491 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f699f33-c208-4bb3-b12f-019c0fbc5bab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:17.076393 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f699f33-c208-4bb3-b12f-019c0fbc5bab service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:17.076686 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f699f33-c208-4bb3-b12f-019c0fbc5bab service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:17.076981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f699f33-c208-4bb3-b12f-019c0fbc5bab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:17.089841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f699f33-c208-4bb3-b12f-019c0fbc5bab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:17.091345 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f699f33-c208-4bb3-b12f-019c0fbc5bab service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:17.103682 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1232/2454] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:39:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:39:17.437723 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7505e-502d-41fb-96af-21794c97d383 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:17.439861 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dcb7505e-502d-41fb-96af-21794c97d383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:17.458045 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7505e-502d-41fb-96af-21794c97d383 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:17.473659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e33a6-dc5f-4123-af22-f7bec756aa03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:17.475049 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e33a6-dc5f-4123-af22-f7bec756aa03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:17.475435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e33a6-dc5f-4123-af22-f7bec756aa03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:17.475811 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64e33a6-dc5f-4123-af22-f7bec756aa03 admin admin] PATH_INFO: `/v3/projects/522f8edfe7114a028b359c8dd9faffbe` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:17.478945 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e33a6-dc5f-4123-af22-f7bec756aa03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=522f8edfe7114a028b359c8dd9faffbe)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:17.482092 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64e33a6-dc5f-4123-af22-f7bec756aa03 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:17.557588 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e64e33a6-dc5f-4123-af22-f7bec756aa03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '522f8edfe7114a028b359c8dd9faffbe', 'request_id': 'req-e64e33a6-dc5f-4123-af22-f7bec756aa03'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:17.567936 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e64e33a6-dc5f-4123-af22-f7bec756aa03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '522f8edfe7114a028b359c8dd9faffbe', 'request_id': 'req-e64e33a6-dc5f-4123-af22-f7bec756aa03'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:17.574142 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e64e33a6-dc5f-4123-af22-f7bec756aa03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '522f8edfe7114a028b359c8dd9faffbe', 'request_id': 'req-e64e33a6-dc5f-4123-af22-f7bec756aa03'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:17.592023 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e64e33a6-dc5f-4123-af22-f7bec756aa03 admin admin] The token cache is being invalidate because project 522f8edfe7114a028b359c8dd9faffbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:39:17.592301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e64e33a6-dc5f-4123-af22-f7bec756aa03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:17.592966 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e64e33a6-dc5f-4123-af22-f7bec756aa03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:17.596143 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1223/2455] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:39:17 2022] DELETE /identity/v3/projects/522f8edfe7114a028b359c8dd9faffbe => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:39:17.599938 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f699f33-c208-4bb3-b12f-019c0fbc5bab service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:17.612311 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f699f33-c208-4bb3-b12f-019c0fbc5bab service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:17.666642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:17.682893 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf06ccb-d27e-424e-a075-6dcbefe5c75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:17.684491 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf06ccb-d27e-424e-a075-6dcbefe5c75c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:17.684945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf06ccb-d27e-424e-a075-6dcbefe5c75c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:17.685227 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf06ccb-d27e-424e-a075-6dcbefe5c75c admin admin] PATH_INFO: `/v3/users/939abacb103c49e0a190e1bf59ebd31f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:17.686501 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf06ccb-d27e-424e-a075-6dcbefe5c75c admin admin] RBAC: Authorizing `identity:delete_user(user_id=939abacb103c49e0a190e1bf59ebd31f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:17.688750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf06ccb-d27e-424e-a075-6dcbefe5c75c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:17.742784 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-baf06ccb-d27e-424e-a075-6dcbefe5c75c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '939abacb103c49e0a190e1bf59ebd31f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:17.743701 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-baf06ccb-d27e-424e-a075-6dcbefe5c75c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '939abacb103c49e0a190e1bf59ebd31f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:17.763703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-baf06ccb-d27e-424e-a075-6dcbefe5c75c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '939abacb103c49e0a190e1bf59ebd31f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:17.766092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-baf06ccb-d27e-424e-a075-6dcbefe5c75c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '939abacb103c49e0a190e1bf59ebd31f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:17.774157 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-baf06ccb-d27e-424e-a075-6dcbefe5c75c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '939abacb103c49e0a190e1bf59ebd31f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:17.802231 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1233/2456] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:39:17 2022] DELETE /identity/v3/users/939abacb103c49e0a190e1bf59ebd31f => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:39:18.131863 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e33a6-dc5f-4123-af22-f7bec756aa03 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:18.136234 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e64e33a6-dc5f-4123-af22-f7bec756aa03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:18.152828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64e33a6-dc5f-4123-af22-f7bec756aa03 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:18.244754 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a0d7a-6a1c-497b-9087-f8b59d687a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:18.245767 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a0d7a-6a1c-497b-9087-f8b59d687a9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:18.246126 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a0d7a-6a1c-497b-9087-f8b59d687a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:18.246502 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a0d7a-6a1c-497b-9087-f8b59d687a9c admin admin] PATH_INFO: `/v3/projects/2591c013916e4234a28200f427fa2294` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:18.247909 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a0d7a-6a1c-497b-9087-f8b59d687a9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2591c013916e4234a28200f427fa2294)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:18.250002 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a0d7a-6a1c-497b-9087-f8b59d687a9c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:18.338665 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-588a0d7a-6a1c-497b-9087-f8b59d687a9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2591c013916e4234a28200f427fa2294', 'request_id': 'req-588a0d7a-6a1c-497b-9087-f8b59d687a9c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:18.347536 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-588a0d7a-6a1c-497b-9087-f8b59d687a9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2591c013916e4234a28200f427fa2294', 'request_id': 'req-588a0d7a-6a1c-497b-9087-f8b59d687a9c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:18.354564 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-588a0d7a-6a1c-497b-9087-f8b59d687a9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2591c013916e4234a28200f427fa2294', 'request_id': 'req-588a0d7a-6a1c-497b-9087-f8b59d687a9c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:18.358533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:18.380198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-588a0d7a-6a1c-497b-9087-f8b59d687a9c admin admin] The token cache is being invalidate because project 2591c013916e4234a28200f427fa2294 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:39:18.380673 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-588a0d7a-6a1c-497b-9087-f8b59d687a9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:18.381432 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-588a0d7a-6a1c-497b-9087-f8b59d687a9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:18.383220 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1224/2457] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:39:18 2022] DELETE /identity/v3/projects/2591c013916e4234a28200f427fa2294 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:39:18.390446 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79755e5-95af-4add-86f8-93a7de489e76 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:18.390793 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79755e5-95af-4add-86f8-93a7de489e76 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:18.391737 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79755e5-95af-4add-86f8-93a7de489e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:18.422133 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a79755e5-95af-4add-86f8-93a7de489e76 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:39:18.426561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a79755e5-95af-4add-86f8-93a7de489e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:18.534246 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1234/2458] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:39:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:39:18.542899 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447c17c-37a2-4ab6-991f-b4e6b76b8860 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:18.543189 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447c17c-37a2-4ab6-991f-b4e6b76b8860 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:18.543479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447c17c-37a2-4ab6-991f-b4e6b76b8860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:18.575162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a447c17c-37a2-4ab6-991f-b4e6b76b8860 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:39:18.576382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a447c17c-37a2-4ab6-991f-b4e6b76b8860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:18.586064 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1225/2459] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:39:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:39:18.590844 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79755e5-95af-4add-86f8-93a7de489e76 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:18.601246 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79755e5-95af-4add-86f8-93a7de489e76 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:18.610261 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc58020-0cd6-4c9f-888c-3d0cf6378446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:18.611076 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc58020-0cd6-4c9f-888c-3d0cf6378446 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:18.611365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc58020-0cd6-4c9f-888c-3d0cf6378446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:18.611787 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc58020-0cd6-4c9f-888c-3d0cf6378446 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:18.612529 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc58020-0cd6-4c9f-888c-3d0cf6378446 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:39:18.612841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc58020-0cd6-4c9f-888c-3d0cf6378446 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:18.614673 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc58020-0cd6-4c9f-888c-3d0cf6378446 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:18.628616 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1235/2460] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:39:18.638586 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447c17c-37a2-4ab6-991f-b4e6b76b8860 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:18.649612 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447c17c-37a2-4ab6-991f-b4e6b76b8860 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:18.654577 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4433ae-9159-48dd-b0b1-f9407a2562bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:18.655277 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4433ae-9159-48dd-b0b1-f9407a2562bc admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:18.655565 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4433ae-9159-48dd-b0b1-f9407a2562bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:18.655845 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4433ae-9159-48dd-b0b1-f9407a2562bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:18.656617 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4433ae-9159-48dd-b0b1-f9407a2562bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:18.658704 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4433ae-9159-48dd-b0b1-f9407a2562bc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:18.690470 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1226/2461] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:39:18 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:39:18.694099 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc58020-0cd6-4c9f-888c-3d0cf6378446 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:18.710862 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc58020-0cd6-4c9f-888c-3d0cf6378446 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:18.770104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:18.780072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f724fe-0ff7-4b1d-a563-37c5476ef20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:18.782641 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f724fe-0ff7-4b1d-a563-37c5476ef20a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:18.782641 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f724fe-0ff7-4b1d-a563-37c5476ef20a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:18.782641 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f724fe-0ff7-4b1d-a563-37c5476ef20a admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:18.782996 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f724fe-0ff7-4b1d-a563-37c5476ef20a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:18.788201 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f724fe-0ff7-4b1d-a563-37c5476ef20a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:18.836206 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1236/2462] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:39:18 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:39:18.840350 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4433ae-9159-48dd-b0b1-f9407a2562bc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:18.851047 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4433ae-9159-48dd-b0b1-f9407a2562bc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:18.855597 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9fde2-81b4-4418-a4c8-310f49e17a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:18.856254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9fde2-81b4-4418-a4c8-310f49e17a9d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:18.856536 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9fde2-81b4-4418-a4c8-310f49e17a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:18.856814 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9fde2-81b4-4418-a4c8-310f49e17a9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:18.857532 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa9fde2-81b4-4418-a4c8-310f49e17a9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:18.859170 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa9fde2-81b4-4418-a4c8-310f49e17a9d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:18.878451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1227/2463] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:39:18.881219 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f724fe-0ff7-4b1d-a563-37c5476ef20a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:18.893008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f724fe-0ff7-4b1d-a563-37c5476ef20a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:18.898344 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733ee8e3-17f1-4cf8-bd2a-a35a5e20fe71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:18.899450 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ee8e3-17f1-4cf8-bd2a-a35a5e20fe71 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:18.899845 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ee8e3-17f1-4cf8-bd2a-a35a5e20fe71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:18.900232 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ee8e3-17f1-4cf8-bd2a-a35a5e20fe71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:18.901069 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733ee8e3-17f1-4cf8-bd2a-a35a5e20fe71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:18.902818 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733ee8e3-17f1-4cf8-bd2a-a35a5e20fe71 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:18.915431 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1237/2464] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:39:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:39:18.918575 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9fde2-81b4-4418-a4c8-310f49e17a9d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:18.931108 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9fde2-81b4-4418-a4c8-310f49e17a9d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:18.935888 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528dc297-6ceb-4bbb-99df-41dabed4ea19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:18.936959 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528dc297-6ceb-4bbb-99df-41dabed4ea19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:18.937314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528dc297-6ceb-4bbb-99df-41dabed4ea19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:18.938245 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528dc297-6ceb-4bbb-99df-41dabed4ea19 admin admin] PATH_INFO: `/v3/projects/5366e779b2aa4256a2496692af000b91/users/feac3312d95944db8e019f417861a72c/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:18.963060 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528dc297-6ceb-4bbb-99df-41dabed4ea19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5366e779b2aa4256a2496692af000b91, user_id=feac3312d95944db8e019f417861a72c, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:18.967161 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528dc297-6ceb-4bbb-99df-41dabed4ea19 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:18.985137 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1228/2465] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:39:18 2022] PUT /identity/v3/projects/5366e779b2aa4256a2496692af000b91/users/feac3312d95944db8e019f417861a72c/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:39:18.995491 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98c958-b6f0-43f7-81e1-408a40203552 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:18.996227 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98c958-b6f0-43f7-81e1-408a40203552 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:18.996940 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98c958-b6f0-43f7-81e1-408a40203552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:19.028597 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ef98c958-b6f0-43f7-81e1-408a40203552 None None] MFA Rules not processed for user `feac3312d95944db8e019f417861a72c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:39:19.029740 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ef98c958-b6f0-43f7-81e1-408a40203552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:19.111861 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1238/2466] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:39:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:39:19.116289 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528dc297-6ceb-4bbb-99df-41dabed4ea19 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:19.190965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528dc297-6ceb-4bbb-99df-41dabed4ea19 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:19.195824 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1cb69-0d67-40a2-ae0e-8fc5048fcbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:19.196505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1cb69-0d67-40a2-ae0e-8fc5048fcbc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:19.196790 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1cb69-0d67-40a2-ae0e-8fc5048fcbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:19.197113 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1cb69-0d67-40a2-ae0e-8fc5048fcbc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:19.197934 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a1cb69-0d67-40a2-ae0e-8fc5048fcbc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:19.200039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a1cb69-0d67-40a2-ae0e-8fc5048fcbc3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:19.238241 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1229/2467] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:39:19 2022] POST /identity/v3/projects => generated 377 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:39:19.239714 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98c958-b6f0-43f7-81e1-408a40203552 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:19.254603 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98c958-b6f0-43f7-81e1-408a40203552 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:19.332746 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e559d-97a1-4344-87ce-db5e661d2354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:19.333444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e559d-97a1-4344-87ce-db5e661d2354 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:19.333814 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e559d-97a1-4344-87ce-db5e661d2354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:19.334161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e559d-97a1-4344-87ce-db5e661d2354 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:19.334994 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e559d-97a1-4344-87ce-db5e661d2354 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:19.337151 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e559d-97a1-4344-87ce-db5e661d2354 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:19.367962 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1239/2468] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:39:19 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:39:19.369056 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1cb69-0d67-40a2-ae0e-8fc5048fcbc3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:19.381736 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1cb69-0d67-40a2-ae0e-8fc5048fcbc3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:19.388750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad6d1cc-cf1b-4503-a7f4-fb52d66c6f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:19.389448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6d1cc-cf1b-4503-a7f4-fb52d66c6f2c admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:19.389737 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6d1cc-cf1b-4503-a7f4-fb52d66c6f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:19.390025 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6d1cc-cf1b-4503-a7f4-fb52d66c6f2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:19.391990 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad6d1cc-cf1b-4503-a7f4-fb52d66c6f2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:19.393620 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad6d1cc-cf1b-4503-a7f4-fb52d66c6f2c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:19.406472 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1230/2469] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:39:19.411278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e559d-97a1-4344-87ce-db5e661d2354 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:19.422752 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e559d-97a1-4344-87ce-db5e661d2354 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:19.427648 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8bc98f-1d81-4afd-aeaf-4a0f8ceed975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:19.428317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8bc98f-1d81-4afd-aeaf-4a0f8ceed975 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:19.428606 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8bc98f-1d81-4afd-aeaf-4a0f8ceed975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:19.428886 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8bc98f-1d81-4afd-aeaf-4a0f8ceed975 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:19.432170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8bc98f-1d81-4afd-aeaf-4a0f8ceed975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:19.432170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8bc98f-1d81-4afd-aeaf-4a0f8ceed975 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:19.443487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1240/2470] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:39:19.447368 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad6d1cc-cf1b-4503-a7f4-fb52d66c6f2c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:19.460769 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad6d1cc-cf1b-4503-a7f4-fb52d66c6f2c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:19.467328 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ab2d6-beec-42f6-be07-0bafb444f86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:19.468226 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ab2d6-beec-42f6-be07-0bafb444f86f admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:19.468513 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ab2d6-beec-42f6-be07-0bafb444f86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:19.468907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ab2d6-beec-42f6-be07-0bafb444f86f admin admin] PATH_INFO: `/v3/projects/21c179e2ef7a48bdb32ad6b2d2afe20f/users/d2ac384af0094007b72da87f5a55cd2c/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:19.488141 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ab2d6-beec-42f6-be07-0bafb444f86f admin admin] RBAC: Authorizing `identity:create_grant(project_id=21c179e2ef7a48bdb32ad6b2d2afe20f, user_id=d2ac384af0094007b72da87f5a55cd2c, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:19.490911 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ab2d6-beec-42f6-be07-0bafb444f86f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:19.508926 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1231/2471] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:39:19 2022] PUT /identity/v3/projects/21c179e2ef7a48bdb32ad6b2d2afe20f/users/d2ac384af0094007b72da87f5a55cd2c/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:39:19.520152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4dc3d3-aa90-4ad4-8ce0-a6249e1edd60 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:19.520689 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4dc3d3-aa90-4ad4-8ce0-a6249e1edd60 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:19.520984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4dc3d3-aa90-4ad4-8ce0-a6249e1edd60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:19.560161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6d4dc3d3-aa90-4ad4-8ce0-a6249e1edd60 None None] MFA Rules not processed for user `d2ac384af0094007b72da87f5a55cd2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:39:19.560161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6d4dc3d3-aa90-4ad4-8ce0-a6249e1edd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:19.597871 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:19.646476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1241/2472] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:39:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:39:19.650029 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ab2d6-beec-42f6-be07-0bafb444f86f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:19.717431 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:19.743028 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ab2d6-beec-42f6-be07-0bafb444f86f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:19.749276 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139e475-2b82-4087-8f52-c9ac3720dadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:19.749993 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139e475-2b82-4087-8f52-c9ac3720dadf admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:19.750285 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139e475-2b82-4087-8f52-c9ac3720dadf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:19.750607 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139e475-2b82-4087-8f52-c9ac3720dadf admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:19.751321 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8139e475-2b82-4087-8f52-c9ac3720dadf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:19.753275 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8139e475-2b82-4087-8f52-c9ac3720dadf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:19.784050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1232/2473] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:39:19 2022] POST /identity/v3/projects => generated 379 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:39:19.788008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4dc3d3-aa90-4ad4-8ce0-a6249e1edd60 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:19.802619 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4dc3d3-aa90-4ad4-8ce0-a6249e1edd60 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:19.871005 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b295d-3d6c-4247-a26a-a23b294636c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:19.871811 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b295d-3d6c-4247-a26a-a23b294636c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:19.872283 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b295d-3d6c-4247-a26a-a23b294636c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:19.872961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09b295d-3d6c-4247-a26a-a23b294636c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:19.873798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b295d-3d6c-4247-a26a-a23b294636c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:19.876791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09b295d-3d6c-4247-a26a-a23b294636c6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:19.904343 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1242/2474] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:39:19 2022] POST /identity/v3/users => generated 296 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:39:19.908762 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139e475-2b82-4087-8f52-c9ac3720dadf admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:19.922778 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139e475-2b82-4087-8f52-c9ac3720dadf admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:19.928017 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297dbd5f-78d9-4133-a0ae-bea24d8958b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:19.928694 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297dbd5f-78d9-4133-a0ae-bea24d8958b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:19.928979 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297dbd5f-78d9-4133-a0ae-bea24d8958b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:19.929258 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297dbd5f-78d9-4133-a0ae-bea24d8958b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:19.931107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297dbd5f-78d9-4133-a0ae-bea24d8958b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:19.934644 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297dbd5f-78d9-4133-a0ae-bea24d8958b1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:19.948334 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1233/2475] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:39:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:39:19.954581 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b295d-3d6c-4247-a26a-a23b294636c6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:19.965564 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09b295d-3d6c-4247-a26a-a23b294636c6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:19.972559 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9fcf6b-0d45-445c-9180-c2fa23bc499b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:19.973478 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fcf6b-0d45-445c-9180-c2fa23bc499b admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:19.973763 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fcf6b-0d45-445c-9180-c2fa23bc499b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:19.974043 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fcf6b-0d45-445c-9180-c2fa23bc499b admin admin] PATH_INFO: `/v3/projects/b38427d547784b07ba95afe6c5e7fdc1/users/eb855280754545bdae59c184b04aaf08/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:19.993589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9fcf6b-0d45-445c-9180-c2fa23bc499b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b38427d547784b07ba95afe6c5e7fdc1, user_id=eb855280754545bdae59c184b04aaf08, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:19.996611 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9fcf6b-0d45-445c-9180-c2fa23bc499b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:20.014217 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1243/2476] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:39:19 2022] PUT /identity/v3/projects/b38427d547784b07ba95afe6c5e7fdc1/users/eb855280754545bdae59c184b04aaf08/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:39:20.021686 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297dbd5f-78d9-4133-a0ae-bea24d8958b1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:20.032498 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297dbd5f-78d9-4133-a0ae-bea24d8958b1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:20.107767 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3ab8a-b6ec-473e-9657-0b4ec3ea487a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:20.108674 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3ab8a-b6ec-473e-9657-0b4ec3ea487a admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:20.109078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3ab8a-b6ec-473e-9657-0b4ec3ea487a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:20.109483 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3ab8a-b6ec-473e-9657-0b4ec3ea487a admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:20.114228 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed3ab8a-b6ec-473e-9657-0b4ec3ea487a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:20.114228 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed3ab8a-b6ec-473e-9657-0b4ec3ea487a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:20.128108 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1234/2477] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:39:20.132387 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9fcf6b-0d45-445c-9180-c2fa23bc499b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:20.143509 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9fcf6b-0d45-445c-9180-c2fa23bc499b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:20.149864 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e27d17-bf6a-4e23-8265-d450dae081d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:20.150808 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e27d17-bf6a-4e23-8265-d450dae081d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:20.151093 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e27d17-bf6a-4e23-8265-d450dae081d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:20.151368 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e27d17-bf6a-4e23-8265-d450dae081d2 admin admin] PATH_INFO: `/v3/projects/b38427d547784b07ba95afe6c5e7fdc1/users/eb855280754545bdae59c184b04aaf08/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:20.153383 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e27d17-bf6a-4e23-8265-d450dae081d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b38427d547784b07ba95afe6c5e7fdc1, user_id=eb855280754545bdae59c184b04aaf08, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:20.156258 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e27d17-bf6a-4e23-8265-d450dae081d2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:20.170474 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1244/2478] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:39:20 2022] PUT /identity/v3/projects/b38427d547784b07ba95afe6c5e7fdc1/users/eb855280754545bdae59c184b04aaf08/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:39:20.180770 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381c10b-020f-42cc-a651-1b25d03ec5b5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:20.181064 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381c10b-020f-42cc-a651-1b25d03ec5b5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:20.181346 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6381c10b-020f-42cc-a651-1b25d03ec5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:20.220269 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6381c10b-020f-42cc-a651-1b25d03ec5b5 None None] MFA Rules not processed for user `eb855280754545bdae59c184b04aaf08`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:39:20.222477 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6381c10b-020f-42cc-a651-1b25d03ec5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:20.303791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1235/2479] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:39:20 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:39:20.315597 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e27d17-bf6a-4e23-8265-d450dae081d2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:20.317278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-04e27d17-bf6a-4e23-8265-d450dae081d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:20.420748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e27d17-bf6a-4e23-8265-d450dae081d2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:20.429149 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc16e8-e0fb-4bad-b07d-8327a0e83d9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:20.430374 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc16e8-e0fb-4bad-b07d-8327a0e83d9a service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:20.430672 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc16e8-e0fb-4bad-b07d-8327a0e83d9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:20.431182 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc16e8-e0fb-4bad-b07d-8327a0e83d9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:20.445749 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc16e8-e0fb-4bad-b07d-8327a0e83d9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:20.447344 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc16e8-e0fb-4bad-b07d-8327a0e83d9a service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:20.461498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1245/2480] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:39:20 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:39:20.504336 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381c10b-020f-42cc-a651-1b25d03ec5b5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:20.506152 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6381c10b-020f-42cc-a651-1b25d03ec5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:20.518958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6381c10b-020f-42cc-a651-1b25d03ec5b5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:20.525026 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9197ed-939f-4bcf-b427-14ea7ba5497c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:20.525799 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9197ed-939f-4bcf-b427-14ea7ba5497c service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:20.526166 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9197ed-939f-4bcf-b427-14ea7ba5497c service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:20.526540 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9197ed-939f-4bcf-b427-14ea7ba5497c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:20.566065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:20.593209 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9197ed-939f-4bcf-b427-14ea7ba5497c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:20.595148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9197ed-939f-4bcf-b427-14ea7ba5497c service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:20.606257 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1236/2481] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:39:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:39:23.480684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29285d20-1aed-4dfb-91a0-f4d67eff500f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:23.481198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29285d20-1aed-4dfb-91a0-f4d67eff500f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:23.481315 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29285d20-1aed-4dfb-91a0-f4d67eff500f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:23.499187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-29285d20-1aed-4dfb-91a0-f4d67eff500f None None] MFA Rules not processed for user `2af4d4a69e9d4755bff61401f4349c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:39:23.500422 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-29285d20-1aed-4dfb-91a0-f4d67eff500f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:23.523323 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:23.560583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1246/2482] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:39:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:39:23.810903 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9197ed-939f-4bcf-b427-14ea7ba5497c service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:23.885551 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9197ed-939f-4bcf-b427-14ea7ba5497c service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:23.890207 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad13b52-2aa1-4cc0-bfe2-472df2068f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:23.891584 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad13b52-2aa1-4cc0-bfe2-472df2068f89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:23.891866 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad13b52-2aa1-4cc0-bfe2-472df2068f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:23.892142 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad13b52-2aa1-4cc0-bfe2-472df2068f89 admin admin] PATH_INFO: `/v3/users/feac3312d95944db8e019f417861a72c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:23.892267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29285d20-1aed-4dfb-91a0-f4d67eff500f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:23.893345 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad13b52-2aa1-4cc0-bfe2-472df2068f89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=feac3312d95944db8e019f417861a72c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:23.893555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-29285d20-1aed-4dfb-91a0-f4d67eff500f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:23.895559 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad13b52-2aa1-4cc0-bfe2-472df2068f89 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:23.948278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8ad13b52-2aa1-4cc0-bfe2-472df2068f89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feac3312d95944db8e019f417861a72c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:23.957193 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8ad13b52-2aa1-4cc0-bfe2-472df2068f89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feac3312d95944db8e019f417861a72c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:23.963973 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29285d20-1aed-4dfb-91a0-f4d67eff500f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:23.967760 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c4763-df14-48fe-9249-4989bb21ff5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:23.968423 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c4763-df14-48fe-9249-4989bb21ff5d service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:23.969270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c4763-df14-48fe-9249-4989bb21ff5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:23.970009 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c4763-df14-48fe-9249-4989bb21ff5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:23.971558 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-535c4763-df14-48fe-9249-4989bb21ff5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:23.972619 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8ad13b52-2aa1-4cc0-bfe2-472df2068f89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feac3312d95944db8e019f417861a72c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:23.973295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8ad13b52-2aa1-4cc0-bfe2-472df2068f89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feac3312d95944db8e019f417861a72c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:23.973890 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8ad13b52-2aa1-4cc0-bfe2-472df2068f89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feac3312d95944db8e019f417861a72c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:23.982690 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c4763-df14-48fe-9249-4989bb21ff5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:23.984921 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c4763-df14-48fe-9249-4989bb21ff5d service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:23.989073 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1237/2483] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:39:23 2022] DELETE /identity/v3/users/feac3312d95944db8e019f417861a72c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:39:23.993649 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488fac5-f792-466f-aa83-dcbe079e26a5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:23.993992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1247/2484] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:39:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 12:39:23.994115 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488fac5-f792-466f-aa83-dcbe079e26a5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:23.994283 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d488fac5-f792-466f-aa83-dcbe079e26a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:24.012850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d488fac5-f792-466f-aa83-dcbe079e26a5 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:39:24.014958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d488fac5-f792-466f-aa83-dcbe079e26a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:24.072434 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:24.092137 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1238/2485] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:39:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:39:24.102408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c4763-df14-48fe-9249-4989bb21ff5d service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:24.103867 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-535c4763-df14-48fe-9249-4989bb21ff5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:24.114808 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c4763-df14-48fe-9249-4989bb21ff5d service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:24.165794 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22cc1f7-e328-4a51-b037-9c4f0b5e8e3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:24.166551 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cc1f7-e328-4a51-b037-9c4f0b5e8e3b service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:24.166862 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cc1f7-e328-4a51-b037-9c4f0b5e8e3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:24.167154 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cc1f7-e328-4a51-b037-9c4f0b5e8e3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:24.177210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22cc1f7-e328-4a51-b037-9c4f0b5e8e3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:24.178669 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22cc1f7-e328-4a51-b037-9c4f0b5e8e3b service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:24.189020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1248/2486] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:39:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:39:24.324509 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488fac5-f792-466f-aa83-dcbe079e26a5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:24.326820 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d488fac5-f792-466f-aa83-dcbe079e26a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:24.409543 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d488fac5-f792-466f-aa83-dcbe079e26a5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:24.414651 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd2670-3150-446f-b5b9-94bc4bfdfaa0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98135cacb9d4f22846ad33500b9bd1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:24.415297 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd2670-3150-446f-b5b9-94bc4bfdfaa0 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:24.415582 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd2670-3150-446f-b5b9-94bc4bfdfaa0 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:24.415921 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd2670-3150-446f-b5b9-94bc4bfdfaa0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:24.417706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c2bd2670-3150-446f-b5b9-94bc4bfdfaa0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:24.430970 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd2670-3150-446f-b5b9-94bc4bfdfaa0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:24.432568 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd2670-3150-446f-b5b9-94bc4bfdfaa0 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:24.444369 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1239/2487] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:39:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:39:24.512958 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22cc1f7-e328-4a51-b037-9c4f0b5e8e3b service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:24.514609 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f22cc1f7-e328-4a51-b037-9c4f0b5e8e3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:24.528830 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22cc1f7-e328-4a51-b037-9c4f0b5e8e3b service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:24.534680 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c7c69-b6c3-49bf-847e-efb8e6b8c99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:24.535685 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c7c69-b6c3-49bf-847e-efb8e6b8c99d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:24.536033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c7c69-b6c3-49bf-847e-efb8e6b8c99d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:24.536378 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c7c69-b6c3-49bf-847e-efb8e6b8c99d admin admin] PATH_INFO: `/v3/projects/5366e779b2aa4256a2496692af000b91` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:24.538087 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c7c69-b6c3-49bf-847e-efb8e6b8c99d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5366e779b2aa4256a2496692af000b91)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:24.540275 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c7c69-b6c3-49bf-847e-efb8e6b8c99d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:24.622028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b90c7c69-b6c3-49bf-847e-efb8e6b8c99d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5366e779b2aa4256a2496692af000b91', 'request_id': 'req-b90c7c69-b6c3-49bf-847e-efb8e6b8c99d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:24.632149 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b90c7c69-b6c3-49bf-847e-efb8e6b8c99d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5366e779b2aa4256a2496692af000b91', 'request_id': 'req-b90c7c69-b6c3-49bf-847e-efb8e6b8c99d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:24.639101 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b90c7c69-b6c3-49bf-847e-efb8e6b8c99d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5366e779b2aa4256a2496692af000b91', 'request_id': 'req-b90c7c69-b6c3-49bf-847e-efb8e6b8c99d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:24.648828 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:24.661496 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b90c7c69-b6c3-49bf-847e-efb8e6b8c99d admin admin] The token cache is being invalidate because project 5366e779b2aa4256a2496692af000b91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:39:24.661849 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b90c7c69-b6c3-49bf-847e-efb8e6b8c99d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:24.662576 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b90c7c69-b6c3-49bf-847e-efb8e6b8c99d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:24.664648 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1249/2488] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:39:24 2022] DELETE /identity/v3/projects/5366e779b2aa4256a2496692af000b91 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:39:24.669424 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd2670-3150-446f-b5b9-94bc4bfdfaa0 service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:24.681876 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd2670-3150-446f-b5b9-94bc4bfdfaa0 service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:24.752051 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a9a0a-5fc7-4c4f-948a-72f14fbe29ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:24.752961 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a9a0a-5fc7-4c4f-948a-72f14fbe29ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:24.753247 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a9a0a-5fc7-4c4f-948a-72f14fbe29ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:24.753527 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a9a0a-5fc7-4c4f-948a-72f14fbe29ed admin admin] PATH_INFO: `/v3/users/d2ac384af0094007b72da87f5a55cd2c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:24.754756 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35a9a0a-5fc7-4c4f-948a-72f14fbe29ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2ac384af0094007b72da87f5a55cd2c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:24.756687 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35a9a0a-5fc7-4c4f-948a-72f14fbe29ed admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:24.808024 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d35a9a0a-5fc7-4c4f-948a-72f14fbe29ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2ac384af0094007b72da87f5a55cd2c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:24.816822 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d35a9a0a-5fc7-4c4f-948a-72f14fbe29ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2ac384af0094007b72da87f5a55cd2c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:24.830940 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d35a9a0a-5fc7-4c4f-948a-72f14fbe29ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2ac384af0094007b72da87f5a55cd2c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:24.831745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d35a9a0a-5fc7-4c4f-948a-72f14fbe29ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2ac384af0094007b72da87f5a55cd2c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:24.832369 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d35a9a0a-5fc7-4c4f-948a-72f14fbe29ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2ac384af0094007b72da87f5a55cd2c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:24.844970 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1240/2489] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:39:24 2022] DELETE /identity/v3/users/d2ac384af0094007b72da87f5a55cd2c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:39:25.112858 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c7c69-b6c3-49bf-847e-efb8e6b8c99d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:25.114309 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b90c7c69-b6c3-49bf-847e-efb8e6b8c99d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:25.128491 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c7c69-b6c3-49bf-847e-efb8e6b8c99d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:25.186019 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c8c64-f96a-40b4-8f9e-b369b5d27754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:25.186911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c8c64-f96a-40b4-8f9e-b369b5d27754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:25.187195 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c8c64-f96a-40b4-8f9e-b369b5d27754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:25.187487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9c8c64-f96a-40b4-8f9e-b369b5d27754 admin admin] PATH_INFO: `/v3/projects/21c179e2ef7a48bdb32ad6b2d2afe20f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:25.188839 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c8c64-f96a-40b4-8f9e-b369b5d27754 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21c179e2ef7a48bdb32ad6b2d2afe20f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:25.190773 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9c8c64-f96a-40b4-8f9e-b369b5d27754 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:25.235982 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2c9c8c64-f96a-40b4-8f9e-b369b5d27754 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21c179e2ef7a48bdb32ad6b2d2afe20f', 'request_id': 'req-2c9c8c64-f96a-40b4-8f9e-b369b5d27754'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:25.242968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2c9c8c64-f96a-40b4-8f9e-b369b5d27754 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21c179e2ef7a48bdb32ad6b2d2afe20f', 'request_id': 'req-2c9c8c64-f96a-40b4-8f9e-b369b5d27754'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:25.249178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2c9c8c64-f96a-40b4-8f9e-b369b5d27754 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21c179e2ef7a48bdb32ad6b2d2afe20f', 'request_id': 'req-2c9c8c64-f96a-40b4-8f9e-b369b5d27754'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:25.260545 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2c9c8c64-f96a-40b4-8f9e-b369b5d27754 admin admin] The token cache is being invalidate because project 21c179e2ef7a48bdb32ad6b2d2afe20f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:39:25.260831 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2c9c8c64-f96a-40b4-8f9e-b369b5d27754 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:25.261461 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2c9c8c64-f96a-40b4-8f9e-b369b5d27754 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:25.263031 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1250/2490] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:39:25 2022] DELETE /identity/v3/projects/21c179e2ef7a48bdb32ad6b2d2afe20f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:39:25.267320 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a9a0a-5fc7-4c4f-948a-72f14fbe29ed admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:25.268734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d35a9a0a-5fc7-4c4f-948a-72f14fbe29ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:25.277068 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:25.284059 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a9a0a-5fc7-4c4f-948a-72f14fbe29ed admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:25.346892 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee297dd7-1735-40ef-b5d9-98c4b1a2beee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:25.347793 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee297dd7-1735-40ef-b5d9-98c4b1a2beee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:25.348074 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee297dd7-1735-40ef-b5d9-98c4b1a2beee admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:25.348357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee297dd7-1735-40ef-b5d9-98c4b1a2beee admin admin] PATH_INFO: `/v3/users/eb855280754545bdae59c184b04aaf08` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:25.349724 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee297dd7-1735-40ef-b5d9-98c4b1a2beee admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb855280754545bdae59c184b04aaf08)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:25.351707 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee297dd7-1735-40ef-b5d9-98c4b1a2beee admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:25.398510 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ee297dd7-1735-40ef-b5d9-98c4b1a2beee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb855280754545bdae59c184b04aaf08'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:25.405716 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ee297dd7-1735-40ef-b5d9-98c4b1a2beee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb855280754545bdae59c184b04aaf08'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:25.418982 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ee297dd7-1735-40ef-b5d9-98c4b1a2beee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb855280754545bdae59c184b04aaf08'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:25.419694 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ee297dd7-1735-40ef-b5d9-98c4b1a2beee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb855280754545bdae59c184b04aaf08'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:25.420369 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ee297dd7-1735-40ef-b5d9-98c4b1a2beee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb855280754545bdae59c184b04aaf08'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:25.433359 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1241/2491] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:39:25 2022] DELETE /identity/v3/users/eb855280754545bdae59c184b04aaf08 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:39:26.022123 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c8c64-f96a-40b4-8f9e-b369b5d27754 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:26.024013 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2c9c8c64-f96a-40b4-8f9e-b369b5d27754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:26.040270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9c8c64-f96a-40b4-8f9e-b369b5d27754 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:26.102878 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074199e6-9555-49c5-9129-1df2793bd6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:26.103754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074199e6-9555-49c5-9129-1df2793bd6d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:26.104042 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074199e6-9555-49c5-9129-1df2793bd6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:26.104326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074199e6-9555-49c5-9129-1df2793bd6d4 admin admin] PATH_INFO: `/v3/projects/b38427d547784b07ba95afe6c5e7fdc1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:26.105590 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074199e6-9555-49c5-9129-1df2793bd6d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b38427d547784b07ba95afe6c5e7fdc1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:26.107594 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074199e6-9555-49c5-9129-1df2793bd6d4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:26.153349 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:26.170444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-074199e6-9555-49c5-9129-1df2793bd6d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b38427d547784b07ba95afe6c5e7fdc1', 'request_id': 'req-074199e6-9555-49c5-9129-1df2793bd6d4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:26.181207 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-074199e6-9555-49c5-9129-1df2793bd6d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b38427d547784b07ba95afe6c5e7fdc1', 'request_id': 'req-074199e6-9555-49c5-9129-1df2793bd6d4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:26.191617 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-074199e6-9555-49c5-9129-1df2793bd6d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b38427d547784b07ba95afe6c5e7fdc1', 'request_id': 'req-074199e6-9555-49c5-9129-1df2793bd6d4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:26.205376 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-074199e6-9555-49c5-9129-1df2793bd6d4 admin admin] The token cache is being invalidate because project b38427d547784b07ba95afe6c5e7fdc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:39:26.205666 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-074199e6-9555-49c5-9129-1df2793bd6d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:26.206355 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-074199e6-9555-49c5-9129-1df2793bd6d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:26.212619 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1251/2492] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:39:26 2022] DELETE /identity/v3/projects/b38427d547784b07ba95afe6c5e7fdc1 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:39:26.214351 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497a379-a3e0-4dc4-836c-6a34af84d3bd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:26.214902 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497a379-a3e0-4dc4-836c-6a34af84d3bd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:26.215621 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497a379-a3e0-4dc4-836c-6a34af84d3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:26.242850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8497a379-a3e0-4dc4-836c-6a34af84d3bd None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:39:26.244522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8497a379-a3e0-4dc4-836c-6a34af84d3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:26.328913 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1242/2493] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:39:26.337291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c91bb8-26c1-4f1b-b4e9-2280c637959f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:26.337581 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c91bb8-26c1-4f1b-b4e9-2280c637959f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:26.337885 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c91bb8-26c1-4f1b-b4e9-2280c637959f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:26.371087 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f6c91bb8-26c1-4f1b-b4e9-2280c637959f None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:39:26.372305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f6c91bb8-26c1-4f1b-b4e9-2280c637959f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:26.384732 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1252/2494] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:39:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:39:26.385648 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497a379-a3e0-4dc4-836c-6a34af84d3bd None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:26.397274 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497a379-a3e0-4dc4-836c-6a34af84d3bd None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:26.402089 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196240b7-1b61-4a1c-b7cf-10bab6a64517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:26.402919 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196240b7-1b61-4a1c-b7cf-10bab6a64517 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:26.403207 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196240b7-1b61-4a1c-b7cf-10bab6a64517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:26.403483 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196240b7-1b61-4a1c-b7cf-10bab6a64517 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:26.404212 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196240b7-1b61-4a1c-b7cf-10bab6a64517 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:39:26.404518 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196240b7-1b61-4a1c-b7cf-10bab6a64517 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:26.406934 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196240b7-1b61-4a1c-b7cf-10bab6a64517 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:26.426099 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1243/2495] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:39:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:39:26.427563 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c91bb8-26c1-4f1b-b4e9-2280c637959f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:26.438707 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c91bb8-26c1-4f1b-b4e9-2280c637959f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:26.443899 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389af704-5061-4561-938f-6483ae30d5a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:26.444614 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389af704-5061-4561-938f-6483ae30d5a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:26.444877 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389af704-5061-4561-938f-6483ae30d5a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:26.445154 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389af704-5061-4561-938f-6483ae30d5a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:26.445848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389af704-5061-4561-938f-6483ae30d5a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:26.447927 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389af704-5061-4561-938f-6483ae30d5a5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:26.484805 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1253/2496] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:39:26 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:39:26.486835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196240b7-1b61-4a1c-b7cf-10bab6a64517 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:26.497506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196240b7-1b61-4a1c-b7cf-10bab6a64517 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:26.502918 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:26.569982 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9e99e-89a7-4016-98fe-b3c8611e0d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:26.570825 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9e99e-89a7-4016-98fe-b3c8611e0d23 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:26.571216 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9e99e-89a7-4016-98fe-b3c8611e0d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:26.571769 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9e99e-89a7-4016-98fe-b3c8611e0d23 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:26.572585 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9e99e-89a7-4016-98fe-b3c8611e0d23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:26.575787 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9e99e-89a7-4016-98fe-b3c8611e0d23 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:26.602589 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1244/2497] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:39:26 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:39:26.606779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389af704-5061-4561-938f-6483ae30d5a5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:26.621872 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389af704-5061-4561-938f-6483ae30d5a5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:26.629223 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706c934-33f2-438c-ad98-38968e1e28f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:26.630255 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706c934-33f2-438c-ad98-38968e1e28f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:26.630815 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706c934-33f2-438c-ad98-38968e1e28f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:26.631262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6706c934-33f2-438c-ad98-38968e1e28f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:26.632169 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706c934-33f2-438c-ad98-38968e1e28f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:26.633997 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6706c934-33f2-438c-ad98-38968e1e28f0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:26.654178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1254/2498] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:39:26.655347 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9e99e-89a7-4016-98fe-b3c8611e0d23 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:26.673878 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9e99e-89a7-4016-98fe-b3c8611e0d23 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:26.680267 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc80403-ebd9-403f-a5f7-a3f3bac9626b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:26.680627 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc80403-ebd9-403f-a5f7-a3f3bac9626b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:26.680905 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc80403-ebd9-403f-a5f7-a3f3bac9626b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:26.681615 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc80403-ebd9-403f-a5f7-a3f3bac9626b admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:26.682688 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc80403-ebd9-403f-a5f7-a3f3bac9626b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:26.684712 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc80403-ebd9-403f-a5f7-a3f3bac9626b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:26.701722 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1245/2499] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:39:26 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:39:26.706588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706c934-33f2-438c-ad98-38968e1e28f0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:26.718805 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6706c934-33f2-438c-ad98-38968e1e28f0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:26.729962 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5310664b-808a-44f6-a73a-668197c6d5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:26.732373 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310664b-808a-44f6-a73a-668197c6d5ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:26.732668 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310664b-808a-44f6-a73a-668197c6d5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:26.732951 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310664b-808a-44f6-a73a-668197c6d5ca admin admin] PATH_INFO: `/v3/projects/aa727960253e41be8599229e2bc9994c/users/ad5461a99edb4281ba66f3987aacb973/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:26.763815 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5310664b-808a-44f6-a73a-668197c6d5ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa727960253e41be8599229e2bc9994c, user_id=ad5461a99edb4281ba66f3987aacb973, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:26.767420 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5310664b-808a-44f6-a73a-668197c6d5ca admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:26.783858 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1255/2500] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:39:26 2022] PUT /identity/v3/projects/aa727960253e41be8599229e2bc9994c/users/ad5461a99edb4281ba66f3987aacb973/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:39:26.796158 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c382eb0-ace6-4212-831a-9e5fbeea1fb2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:26.796845 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c382eb0-ace6-4212-831a-9e5fbeea1fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:26.797225 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c382eb0-ace6-4212-831a-9e5fbeea1fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:26.826603 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8c382eb0-ace6-4212-831a-9e5fbeea1fb2 None None] MFA Rules not processed for user `ad5461a99edb4281ba66f3987aacb973`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:39:26.827984 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8c382eb0-ace6-4212-831a-9e5fbeea1fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:26.911533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1246/2501] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:39:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:39:26.916485 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5310664b-808a-44f6-a73a-668197c6d5ca admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:26.997633 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5310664b-808a-44f6-a73a-668197c6d5ca admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:27.003770 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a2038-67c6-4103-ac30-391b218d1034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:27.004959 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a2038-67c6-4103-ac30-391b218d1034 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:27.004959 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a2038-67c6-4103-ac30-391b218d1034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:27.005170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a2038-67c6-4103-ac30-391b218d1034 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:27.005698 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a2038-67c6-4103-ac30-391b218d1034 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:27.007661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a2038-67c6-4103-ac30-391b218d1034 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:27.043291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1256/2502] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:39:26 2022] POST /identity/v3/projects => generated 381 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:39:27.044625 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c382eb0-ace6-4212-831a-9e5fbeea1fb2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:27.061784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c382eb0-ace6-4212-831a-9e5fbeea1fb2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:27.120023 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc0089-f4b3-4abd-9730-69cfb268e482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:27.120696 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc0089-f4b3-4abd-9730-69cfb268e482 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:27.120975 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc0089-f4b3-4abd-9730-69cfb268e482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:27.121250 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc0089-f4b3-4abd-9730-69cfb268e482 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:27.121953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc0089-f4b3-4abd-9730-69cfb268e482 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:27.123950 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cc0089-f4b3-4abd-9730-69cfb268e482 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:27.147738 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1247/2503] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:39:27 2022] POST /identity/v3/users => generated 297 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:39:27.151450 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a2038-67c6-4103-ac30-391b218d1034 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:27.162312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a2038-67c6-4103-ac30-391b218d1034 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:27.167030 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a1966-92f7-46af-9359-9ffb5e007032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:27.167722 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a1966-92f7-46af-9359-9ffb5e007032 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:27.168012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a1966-92f7-46af-9359-9ffb5e007032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:27.168294 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a1966-92f7-46af-9359-9ffb5e007032 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:27.169014 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a1966-92f7-46af-9359-9ffb5e007032 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:27.170640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a1966-92f7-46af-9359-9ffb5e007032 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:27.183301 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1257/2504] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:39:27.188087 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc0089-f4b3-4abd-9730-69cfb268e482 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:27.201802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc0089-f4b3-4abd-9730-69cfb268e482 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:27.218194 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69155f3-a609-4b78-8448-f951279928e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:27.220202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69155f3-a609-4b78-8448-f951279928e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:27.220675 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69155f3-a609-4b78-8448-f951279928e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:27.221481 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69155f3-a609-4b78-8448-f951279928e2 admin admin] PATH_INFO: `/v3/projects/a38effe847714478b558bacc65296dcc/users/0a242a0ebb974aa783b6c88ec793e506/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:27.225333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:27.242264 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69155f3-a609-4b78-8448-f951279928e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a38effe847714478b558bacc65296dcc, user_id=0a242a0ebb974aa783b6c88ec793e506, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:27.245320 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69155f3-a609-4b78-8448-f951279928e2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:27.261749 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1248/2505] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:39:27 2022] PUT /identity/v3/projects/a38effe847714478b558bacc65296dcc/users/0a242a0ebb974aa783b6c88ec793e506/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:39:27.265601 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a1966-92f7-46af-9359-9ffb5e007032 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:27.276457 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a1966-92f7-46af-9359-9ffb5e007032 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:27.315363 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:27.365539 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879834fe-039f-4ddd-a9c7-403f56345e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:27.366665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879834fe-039f-4ddd-a9c7-403f56345e40 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:27.367078 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879834fe-039f-4ddd-a9c7-403f56345e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:27.367450 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879834fe-039f-4ddd-a9c7-403f56345e40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:27.368284 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879834fe-039f-4ddd-a9c7-403f56345e40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:27.371322 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879834fe-039f-4ddd-a9c7-403f56345e40 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:27.397717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69155f3-a609-4b78-8448-f951279928e2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:27.399783 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1258/2506] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:39:27.412565 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69155f3-a609-4b78-8448-f951279928e2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:27.418765 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0438e6e9-d304-4ba9-bac7-5deb1d13ae8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:27.419886 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438e6e9-d304-4ba9-bac7-5deb1d13ae8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:27.420174 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438e6e9-d304-4ba9-bac7-5deb1d13ae8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:27.420453 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0438e6e9-d304-4ba9-bac7-5deb1d13ae8f admin admin] PATH_INFO: `/v3/projects/a38effe847714478b558bacc65296dcc/users/0a242a0ebb974aa783b6c88ec793e506/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:27.423533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0438e6e9-d304-4ba9-bac7-5deb1d13ae8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a38effe847714478b558bacc65296dcc, user_id=0a242a0ebb974aa783b6c88ec793e506, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:27.426235 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0438e6e9-d304-4ba9-bac7-5deb1d13ae8f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:27.442908 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1249/2507] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:39:27 2022] PUT /identity/v3/projects/a38effe847714478b558bacc65296dcc/users/0a242a0ebb974aa783b6c88ec793e506/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:39:27.454004 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18745e-54e8-434f-beb8-22749365dac6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:27.454296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18745e-54e8-434f-beb8-22749365dac6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:27.454620 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb18745e-54e8-434f-beb8-22749365dac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:27.491385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fb18745e-54e8-434f-beb8-22749365dac6 None None] MFA Rules not processed for user `0a242a0ebb974aa783b6c88ec793e506`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:39:27.493016 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fb18745e-54e8-434f-beb8-22749365dac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:27.586314 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1259/2508] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:39:27 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:39:27.599393 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0438e6e9-d304-4ba9-bac7-5deb1d13ae8f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:27.601092 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0438e6e9-d304-4ba9-bac7-5deb1d13ae8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:27.696029 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0438e6e9-d304-4ba9-bac7-5deb1d13ae8f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:27.701453 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797d66a-f5ed-450d-9cec-c77122d9fe38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:27.702408 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797d66a-f5ed-450d-9cec-c77122d9fe38 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:27.703237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797d66a-f5ed-450d-9cec-c77122d9fe38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:27.703602 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797d66a-f5ed-450d-9cec-c77122d9fe38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:27.715685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2797d66a-f5ed-450d-9cec-c77122d9fe38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:27.717250 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2797d66a-f5ed-450d-9cec-c77122d9fe38 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:27.740109 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1250/2509] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:39:27 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:39:27.819866 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb18745e-54e8-434f-beb8-22749365dac6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:27.830195 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb18745e-54e8-434f-beb8-22749365dac6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:27.834498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27e14c7-a90f-44a2-83d7-e8f34f38e8a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:27.835171 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e14c7-a90f-44a2-83d7-e8f34f38e8a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:27.835497 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e14c7-a90f-44a2-83d7-e8f34f38e8a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:27.835773 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27e14c7-a90f-44a2-83d7-e8f34f38e8a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:27.903014 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e14c7-a90f-44a2-83d7-e8f34f38e8a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:27.904442 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27e14c7-a90f-44a2-83d7-e8f34f38e8a5 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:27.915655 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1260/2510] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:39:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:39:28.240644 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797d66a-f5ed-450d-9cec-c77122d9fe38 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:28.309975 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797d66a-f5ed-450d-9cec-c77122d9fe38 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:28.314699 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6955090-7b37-4e74-9d71-267f83ae7c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:28.315579 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6955090-7b37-4e74-9d71-267f83ae7c1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:28.316818 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6955090-7b37-4e74-9d71-267f83ae7c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:28.316818 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6955090-7b37-4e74-9d71-267f83ae7c1b admin admin] PATH_INFO: `/v3/users/ad5461a99edb4281ba66f3987aacb973` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:28.317464 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6955090-7b37-4e74-9d71-267f83ae7c1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad5461a99edb4281ba66f3987aacb973)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:28.319563 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6955090-7b37-4e74-9d71-267f83ae7c1b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:28.399005 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b6955090-7b37-4e74-9d71-267f83ae7c1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad5461a99edb4281ba66f3987aacb973'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:28.410230 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b6955090-7b37-4e74-9d71-267f83ae7c1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad5461a99edb4281ba66f3987aacb973'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:28.427196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b6955090-7b37-4e74-9d71-267f83ae7c1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad5461a99edb4281ba66f3987aacb973'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:28.428033 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b6955090-7b37-4e74-9d71-267f83ae7c1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad5461a99edb4281ba66f3987aacb973'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:28.428771 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b6955090-7b37-4e74-9d71-267f83ae7c1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad5461a99edb4281ba66f3987aacb973'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:28.431479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:28.452274 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1251/2511] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:39:28 2022] DELETE /identity/v3/users/ad5461a99edb4281ba66f3987aacb973 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:39:28.457225 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7dff80-845d-4978-a031-fe00e708d1a6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:28.457700 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7dff80-845d-4978-a031-fe00e708d1a6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:28.457817 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7dff80-845d-4978-a031-fe00e708d1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:28.477307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4f7dff80-845d-4978-a031-fe00e708d1a6 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:39:28.478576 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4f7dff80-845d-4978-a031-fe00e708d1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:28.535977 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:28.554469 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1261/2512] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:39:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:39:28.562445 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6955090-7b37-4e74-9d71-267f83ae7c1b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:28.563946 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b6955090-7b37-4e74-9d71-267f83ae7c1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:28.625581 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6955090-7b37-4e74-9d71-267f83ae7c1b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:28.629150 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1afe02-9cd5-491b-a908-67390c4005dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:28.629906 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1afe02-9cd5-491b-a908-67390c4005dd service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:28.630546 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1afe02-9cd5-491b-a908-67390c4005dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:28.630546 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1afe02-9cd5-491b-a908-67390c4005dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:28.642522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1afe02-9cd5-491b-a908-67390c4005dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:28.644126 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1afe02-9cd5-491b-a908-67390c4005dd service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:28.654953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1252/2513] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:39:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:39:28.957657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7dff80-845d-4978-a031-fe00e708d1a6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:28.959716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4f7dff80-845d-4978-a031-fe00e708d1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:28.973630 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7dff80-845d-4978-a031-fe00e708d1a6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:28.979642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920285d-d879-40cb-8641-0698eb421338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:28.980526 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920285d-d879-40cb-8641-0698eb421338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:28.980812 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920285d-d879-40cb-8641-0698eb421338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:28.981090 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920285d-d879-40cb-8641-0698eb421338 admin admin] PATH_INFO: `/v3/projects/aa727960253e41be8599229e2bc9994c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:28.982282 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920285d-d879-40cb-8641-0698eb421338 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa727960253e41be8599229e2bc9994c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:28.984983 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920285d-d879-40cb-8641-0698eb421338 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:29.059317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5920285d-d879-40cb-8641-0698eb421338 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa727960253e41be8599229e2bc9994c', 'request_id': 'req-5920285d-d879-40cb-8641-0698eb421338'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:29.068967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5920285d-d879-40cb-8641-0698eb421338 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa727960253e41be8599229e2bc9994c', 'request_id': 'req-5920285d-d879-40cb-8641-0698eb421338'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:29.079837 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5920285d-d879-40cb-8641-0698eb421338 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa727960253e41be8599229e2bc9994c', 'request_id': 'req-5920285d-d879-40cb-8641-0698eb421338'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:29.098785 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5920285d-d879-40cb-8641-0698eb421338 admin admin] The token cache is being invalidate because project aa727960253e41be8599229e2bc9994c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:39:29.099214 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5920285d-d879-40cb-8641-0698eb421338 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:29.099940 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5920285d-d879-40cb-8641-0698eb421338 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:29.101547 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1262/2514] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:39:28 2022] DELETE /identity/v3/projects/aa727960253e41be8599229e2bc9994c => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:39:29.105630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1afe02-9cd5-491b-a908-67390c4005dd service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:29.116089 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1afe02-9cd5-491b-a908-67390c4005dd service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:29.186642 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b99208-0cf9-422f-87d5-35c05b9c7530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:29.187723 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b99208-0cf9-422f-87d5-35c05b9c7530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:29.187866 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b99208-0cf9-422f-87d5-35c05b9c7530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:29.188126 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b99208-0cf9-422f-87d5-35c05b9c7530 admin admin] PATH_INFO: `/v3/users/0a242a0ebb974aa783b6c88ec793e506` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:29.189312 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b99208-0cf9-422f-87d5-35c05b9c7530 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a242a0ebb974aa783b6c88ec793e506)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:29.191281 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b99208-0cf9-422f-87d5-35c05b9c7530 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:29.238861 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-28b99208-0cf9-422f-87d5-35c05b9c7530 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a242a0ebb974aa783b6c88ec793e506'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:29.245884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-28b99208-0cf9-422f-87d5-35c05b9c7530 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a242a0ebb974aa783b6c88ec793e506'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:29.257949 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-28b99208-0cf9-422f-87d5-35c05b9c7530 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a242a0ebb974aa783b6c88ec793e506'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:29.258618 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-28b99208-0cf9-422f-87d5-35c05b9c7530 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a242a0ebb974aa783b6c88ec793e506'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:29.259227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-28b99208-0cf9-422f-87d5-35c05b9c7530 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a242a0ebb974aa783b6c88ec793e506'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:29.272004 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1253/2515] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:39:29 2022] DELETE /identity/v3/users/0a242a0ebb974aa783b6c88ec793e506 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:39:29.538305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920285d-d879-40cb-8641-0698eb421338 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:29.539756 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5920285d-d879-40cb-8641-0698eb421338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:29.552728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920285d-d879-40cb-8641-0698eb421338 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:29.609591 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58341628-ef48-4098-849e-273c5c1d53f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:29.610471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58341628-ef48-4098-849e-273c5c1d53f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:29.610759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58341628-ef48-4098-849e-273c5c1d53f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:29.611034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58341628-ef48-4098-849e-273c5c1d53f1 admin admin] PATH_INFO: `/v3/projects/a38effe847714478b558bacc65296dcc` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:29.612203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58341628-ef48-4098-849e-273c5c1d53f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a38effe847714478b558bacc65296dcc)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:29.614091 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58341628-ef48-4098-849e-273c5c1d53f1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:29.659369 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-58341628-ef48-4098-849e-273c5c1d53f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a38effe847714478b558bacc65296dcc', 'request_id': 'req-58341628-ef48-4098-849e-273c5c1d53f1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:29.666171 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-58341628-ef48-4098-849e-273c5c1d53f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a38effe847714478b558bacc65296dcc', 'request_id': 'req-58341628-ef48-4098-849e-273c5c1d53f1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:29.672665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-58341628-ef48-4098-849e-273c5c1d53f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a38effe847714478b558bacc65296dcc', 'request_id': 'req-58341628-ef48-4098-849e-273c5c1d53f1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:29.682792 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-58341628-ef48-4098-849e-273c5c1d53f1 admin admin] The token cache is being invalidate because project a38effe847714478b558bacc65296dcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:39:29.683079 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-58341628-ef48-4098-849e-273c5c1d53f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:29.683978 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-58341628-ef48-4098-849e-273c5c1d53f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:29.685614 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1263/2516] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:39:29 2022] DELETE /identity/v3/projects/a38effe847714478b558bacc65296dcc => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:39:29.691722 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84374370-e4fe-455a-a0b0-b36886cea7dd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:29.692011 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84374370-e4fe-455a-a0b0-b36886cea7dd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:29.692290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84374370-e4fe-455a-a0b0-b36886cea7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:29.710741 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-84374370-e4fe-455a-a0b0-b36886cea7dd None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:39:29.711951 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-84374370-e4fe-455a-a0b0-b36886cea7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:29.721183 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:29.779055 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1254/2517] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:39:29.786959 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d1610-9e3a-4bd7-b996-70a718ef58dd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:29.787252 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d1610-9e3a-4bd7-b996-70a718ef58dd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:29.787539 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d1610-9e3a-4bd7-b996-70a718ef58dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:29.794915 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:29.808021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-559d1610-9e3a-4bd7-b996-70a718ef58dd None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:39:29.809874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-559d1610-9e3a-4bd7-b996-70a718ef58dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:29.819577 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1264/2518] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:39:29.823807 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84374370-e4fe-455a-a0b0-b36886cea7dd None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:29.834394 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84374370-e4fe-455a-a0b0-b36886cea7dd None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:29.839157 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c947e2e-07ac-451b-ab4f-3a42ddaf2884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:29.839816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c947e2e-07ac-451b-ab4f-3a42ddaf2884 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:29.840105 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c947e2e-07ac-451b-ab4f-3a42ddaf2884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:29.840389 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c947e2e-07ac-451b-ab4f-3a42ddaf2884 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:29.841117 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c947e2e-07ac-451b-ab4f-3a42ddaf2884 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:39:29.841432 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c947e2e-07ac-451b-ab4f-3a42ddaf2884 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:29.843070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c947e2e-07ac-451b-ab4f-3a42ddaf2884 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:29.855310 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1255/2519] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:39:29.859275 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d1610-9e3a-4bd7-b996-70a718ef58dd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:29.868470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d1610-9e3a-4bd7-b996-70a718ef58dd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:29.873187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d187a-6564-4fd2-b325-d8f507d000c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:29.873850 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d187a-6564-4fd2-b325-d8f507d000c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:29.874136 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d187a-6564-4fd2-b325-d8f507d000c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:29.874435 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1d187a-6564-4fd2-b325-d8f507d000c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:29.875722 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d187a-6564-4fd2-b325-d8f507d000c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:29.877619 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1d187a-6564-4fd2-b325-d8f507d000c0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:29.908640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1265/2520] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:39:29 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:39:29.912659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c947e2e-07ac-451b-ab4f-3a42ddaf2884 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:29.925419 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c947e2e-07ac-451b-ab4f-3a42ddaf2884 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:29.984525 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fae2cb-e4ed-416f-9112-67c4e0c8396a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:29.985216 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fae2cb-e4ed-416f-9112-67c4e0c8396a admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:29.985507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fae2cb-e4ed-416f-9112-67c4e0c8396a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:29.985783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fae2cb-e4ed-416f-9112-67c4e0c8396a admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:29.986548 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fae2cb-e4ed-416f-9112-67c4e0c8396a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:29.988488 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fae2cb-e4ed-416f-9112-67c4e0c8396a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:30.013166 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1256/2521] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:39:29 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:39:30.014495 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d187a-6564-4fd2-b325-d8f507d000c0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:30.025480 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1d187a-6564-4fd2-b325-d8f507d000c0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:30.032795 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1f6c5-4494-4884-bd17-d7c370e66e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:30.033604 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1f6c5-4494-4884-bd17-d7c370e66e2f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:30.034036 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1f6c5-4494-4884-bd17-d7c370e66e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:30.034416 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1f6c5-4494-4884-bd17-d7c370e66e2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:30.035284 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced1f6c5-4494-4884-bd17-d7c370e66e2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:30.037194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced1f6c5-4494-4884-bd17-d7c370e66e2f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:30.050048 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc02903-cf28-4ebf-828b-bb57f40bfe90 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:30.053096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc02903-cf28-4ebf-828b-bb57f40bfe90 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:30.053226 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1266/2522] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:39:30.053334 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc02903-cf28-4ebf-828b-bb57f40bfe90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:30.054079 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1f6c5-4494-4884-bd17-d7c370e66e2f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:30.064638 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1f6c5-4494-4884-bd17-d7c370e66e2f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:30.069621 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11785f03-9276-49cd-9cab-2694077b1e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:30.070445 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11785f03-9276-49cd-9cab-2694077b1e38 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:30.070848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11785f03-9276-49cd-9cab-2694077b1e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:30.071218 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-adc02903-cf28-4ebf-828b-bb57f40bfe90 None None] MFA Rules not processed for user `8c10852a728844c6ae0a48e3772d95b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:39:30.071348 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11785f03-9276-49cd-9cab-2694077b1e38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:30.072167 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11785f03-9276-49cd-9cab-2694077b1e38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:30.072427 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-adc02903-cf28-4ebf-828b-bb57f40bfe90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:30.074037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11785f03-9276-49cd-9cab-2694077b1e38 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:30.085031 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1267/2523] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:39:30.088823 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11785f03-9276-49cd-9cab-2694077b1e38 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:30.098827 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11785f03-9276-49cd-9cab-2694077b1e38 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:30.105842 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e55248-ca8f-4345-86ca-9c2d5edfdc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:30.107267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e55248-ca8f-4345-86ca-9c2d5edfdc46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:30.107634 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e55248-ca8f-4345-86ca-9c2d5edfdc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:30.107989 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e55248-ca8f-4345-86ca-9c2d5edfdc46 admin admin] PATH_INFO: `/v3/projects/bc3a9d147c3d449392b4f7ee95c63b72/users/c8457986a5664b258a005a6aedb6e11f/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:30.126793 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e55248-ca8f-4345-86ca-9c2d5edfdc46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc3a9d147c3d449392b4f7ee95c63b72, user_id=c8457986a5664b258a005a6aedb6e11f, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:30.129615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e55248-ca8f-4345-86ca-9c2d5edfdc46 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:30.146253 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1268/2524] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:39:30 2022] PUT /identity/v3/projects/bc3a9d147c3d449392b4f7ee95c63b72/users/c8457986a5664b258a005a6aedb6e11f/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:39:30.154363 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1257/2525] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jun 29 12:39:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:39:30.157319 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff596e-afba-48c4-889f-d89d5c8f8ffa None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:30.157721 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff596e-afba-48c4-889f-d89d5c8f8ffa None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:30.158108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aff596e-afba-48c4-889f-d89d5c8f8ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:30.172084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc02903-cf28-4ebf-828b-bb57f40bfe90 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:30.173966 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-adc02903-cf28-4ebf-828b-bb57f40bfe90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:30.194215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9aff596e-afba-48c4-889f-d89d5c8f8ffa None None] MFA Rules not processed for user `c8457986a5664b258a005a6aedb6e11f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:39:30.195739 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9aff596e-afba-48c4-889f-d89d5c8f8ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:30.222350 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:30.266070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc02903-cf28-4ebf-828b-bb57f40bfe90 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:30.286759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1269/2526] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:39:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:39:30.298826 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff596e-afba-48c4-889f-d89d5c8f8ffa None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:30.306415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9aff596e-afba-48c4-889f-d89d5c8f8ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:30.350766 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46011499-bf31-4515-9958-1b74ca10358d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209b0df45f76404da880a9ae0c637d67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:30.351959 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46011499-bf31-4515-9958-1b74ca10358d service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:30.352315 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46011499-bf31-4515-9958-1b74ca10358d service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:30.352662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46011499-bf31-4515-9958-1b74ca10358d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:30.376186 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aff596e-afba-48c4-889f-d89d5c8f8ffa None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:30.381738 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eba217-cc84-4c2b-a7fa-8776dabba774 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:30.382542 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eba217-cc84-4c2b-a7fa-8776dabba774 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:30.382901 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eba217-cc84-4c2b-a7fa-8776dabba774 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:30.383252 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eba217-cc84-4c2b-a7fa-8776dabba774 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:30.394163 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eba217-cc84-4c2b-a7fa-8776dabba774 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:30.396427 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eba217-cc84-4c2b-a7fa-8776dabba774 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:30.407093 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1270/2527] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:39:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:39:30.433928 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46011499-bf31-4515-9958-1b74ca10358d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:30.435697 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46011499-bf31-4515-9958-1b74ca10358d service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:30.445330 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1258/2528] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 12:39:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 12:39:33.766921 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eba217-cc84-4c2b-a7fa-8776dabba774 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:33.814735 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:33.864620 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eba217-cc84-4c2b-a7fa-8776dabba774 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:33.869833 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db3866-16db-43ad-997a-55b78bf4f69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:33.870783 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db3866-16db-43ad-997a-55b78bf4f69a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:33.871104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db3866-16db-43ad-997a-55b78bf4f69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:33.871388 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db3866-16db-43ad-997a-55b78bf4f69a admin admin] PATH_INFO: `/v3/users/c8457986a5664b258a005a6aedb6e11f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:33.872607 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db3866-16db-43ad-997a-55b78bf4f69a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8457986a5664b258a005a6aedb6e11f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:33.875342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db3866-16db-43ad-997a-55b78bf4f69a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:33.973299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e6db3866-16db-43ad-997a-55b78bf4f69a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8457986a5664b258a005a6aedb6e11f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:33.974079 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e6db3866-16db-43ad-997a-55b78bf4f69a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8457986a5664b258a005a6aedb6e11f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:33.992312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e6db3866-16db-43ad-997a-55b78bf4f69a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8457986a5664b258a005a6aedb6e11f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:33.993240 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e6db3866-16db-43ad-997a-55b78bf4f69a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8457986a5664b258a005a6aedb6e11f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:34.003392 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e6db3866-16db-43ad-997a-55b78bf4f69a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8457986a5664b258a005a6aedb6e11f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:34.019887 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1271/2529] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:39:33 2022] DELETE /identity/v3/users/c8457986a5664b258a005a6aedb6e11f => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:39:34.029745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c2e47-fa79-4ec3-820b-0878b26ccf37 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:34.030558 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c2e47-fa79-4ec3-820b-0878b26ccf37 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:34.030558 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c2e47-fa79-4ec3-820b-0878b26ccf37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:34.054861 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-065c2e47-fa79-4ec3-820b-0878b26ccf37 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:39:34.056259 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-065c2e47-fa79-4ec3-820b-0878b26ccf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:34.133672 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1259/2530] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:39:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:39:34.146231 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db3866-16db-43ad-997a-55b78bf4f69a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:34.147706 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e6db3866-16db-43ad-997a-55b78bf4f69a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:34.228083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db3866-16db-43ad-997a-55b78bf4f69a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:34.231873 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345c1a4-9cbd-4f00-a5d4-ff8525b898a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:34.232537 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345c1a4-9cbd-4f00-a5d4-ff8525b898a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:34.232834 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345c1a4-9cbd-4f00-a5d4-ff8525b898a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:34.233113 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345c1a4-9cbd-4f00-a5d4-ff8525b898a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:34.254524 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9345c1a4-9cbd-4f00-a5d4-ff8525b898a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:34.256540 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9345c1a4-9cbd-4f00-a5d4-ff8525b898a8 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:34.271348 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1272/2531] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:39:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:39:34.689316 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c2e47-fa79-4ec3-820b-0878b26ccf37 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:34.690788 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-065c2e47-fa79-4ec3-820b-0878b26ccf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:34.719090 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c2e47-fa79-4ec3-820b-0878b26ccf37 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:34.733809 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4310c7-7215-4c79-b704-512ce2e3de39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:34.735024 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4310c7-7215-4c79-b704-512ce2e3de39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:34.735630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4310c7-7215-4c79-b704-512ce2e3de39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:34.736163 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4310c7-7215-4c79-b704-512ce2e3de39 admin admin] PATH_INFO: `/v3/projects/bc3a9d147c3d449392b4f7ee95c63b72` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:34.742631 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4310c7-7215-4c79-b704-512ce2e3de39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc3a9d147c3d449392b4f7ee95c63b72)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:34.745055 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4310c7-7215-4c79-b704-512ce2e3de39 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:34.803330 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:34.850641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9e4310c7-7215-4c79-b704-512ce2e3de39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc3a9d147c3d449392b4f7ee95c63b72', 'request_id': 'req-9e4310c7-7215-4c79-b704-512ce2e3de39'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:34.860854 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9e4310c7-7215-4c79-b704-512ce2e3de39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc3a9d147c3d449392b4f7ee95c63b72', 'request_id': 'req-9e4310c7-7215-4c79-b704-512ce2e3de39'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:34.874225 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9e4310c7-7215-4c79-b704-512ce2e3de39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc3a9d147c3d449392b4f7ee95c63b72', 'request_id': 'req-9e4310c7-7215-4c79-b704-512ce2e3de39'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:34.901947 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9e4310c7-7215-4c79-b704-512ce2e3de39 admin admin] The token cache is being invalidate because project bc3a9d147c3d449392b4f7ee95c63b72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:39:34.902246 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9e4310c7-7215-4c79-b704-512ce2e3de39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:34.903026 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9e4310c7-7215-4c79-b704-512ce2e3de39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:34.904989 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1260/2532] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:39:34 2022] DELETE /identity/v3/projects/bc3a9d147c3d449392b4f7ee95c63b72 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:39:34.910443 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11dd034-4740-4e01-9f48-6d12e7d26bed None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:34.911069 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11dd034-4740-4e01-9f48-6d12e7d26bed None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:34.911069 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11dd034-4740-4e01-9f48-6d12e7d26bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:34.930552 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d11dd034-4740-4e01-9f48-6d12e7d26bed None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:39:34.931863 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d11dd034-4740-4e01-9f48-6d12e7d26bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:34.994827 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:35.026099 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1273/2533] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:39:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:39:35.036355 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c647d6-9f61-4853-a59e-fc5269ce5c57 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:35.036355 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c647d6-9f61-4853-a59e-fc5269ce5c57 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:35.036946 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c647d6-9f61-4853-a59e-fc5269ce5c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:35.055480 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-96c647d6-9f61-4853-a59e-fc5269ce5c57 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:39:35.056690 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-96c647d6-9f61-4853-a59e-fc5269ce5c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:35.065321 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1261/2534] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:39:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:39:35.069939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11dd034-4740-4e01-9f48-6d12e7d26bed None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:35.080557 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11dd034-4740-4e01-9f48-6d12e7d26bed None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:35.085392 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b90c2-031e-4cb9-b6a2-50b1a59fba42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:35.086442 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b90c2-031e-4cb9-b6a2-50b1a59fba42 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:35.086753 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b90c2-031e-4cb9-b6a2-50b1a59fba42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:35.087025 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b90c2-031e-4cb9-b6a2-50b1a59fba42 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:35.087759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b90c2-031e-4cb9-b6a2-50b1a59fba42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:39:35.088062 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b90c2-031e-4cb9-b6a2-50b1a59fba42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:35.089667 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3b90c2-031e-4cb9-b6a2-50b1a59fba42 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:35.104293 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1274/2535] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:39:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:39:35.108720 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c647d6-9f61-4853-a59e-fc5269ce5c57 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:35.119789 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c647d6-9f61-4853-a59e-fc5269ce5c57 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:35.124681 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e3bbd-6ae9-4ea7-8961-13c69da53b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:35.125346 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e3bbd-6ae9-4ea7-8961-13c69da53b6e admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:35.125624 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e3bbd-6ae9-4ea7-8961-13c69da53b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:35.125905 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e3bbd-6ae9-4ea7-8961-13c69da53b6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:35.126642 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04e3bbd-6ae9-4ea7-8961-13c69da53b6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:35.129014 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04e3bbd-6ae9-4ea7-8961-13c69da53b6e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:35.185777 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b90c2-031e-4cb9-b6a2-50b1a59fba42 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:35.186706 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1262/2536] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:39:35 2022] POST /identity/v3/projects => generated 391 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:39:35.202708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3b90c2-031e-4cb9-b6a2-50b1a59fba42 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:35.257873 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55251a5f-f699-4f5e-8891-26ef6d252aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:35.258749 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55251a5f-f699-4f5e-8891-26ef6d252aad admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:35.259035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55251a5f-f699-4f5e-8891-26ef6d252aad admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:35.259397 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55251a5f-f699-4f5e-8891-26ef6d252aad admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:35.260143 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55251a5f-f699-4f5e-8891-26ef6d252aad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:35.262275 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55251a5f-f699-4f5e-8891-26ef6d252aad admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:35.292370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1275/2537] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:39:35 2022] POST /identity/v3/users => generated 296 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:39:35.296355 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e3bbd-6ae9-4ea7-8961-13c69da53b6e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:35.307203 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e3bbd-6ae9-4ea7-8961-13c69da53b6e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:35.311965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ba8a2-3bc0-456f-9b62-a97e144446fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:35.312869 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ba8a2-3bc0-456f-9b62-a97e144446fc admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:35.313148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ba8a2-3bc0-456f-9b62-a97e144446fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:35.313464 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85ba8a2-3bc0-456f-9b62-a97e144446fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:35.314157 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ba8a2-3bc0-456f-9b62-a97e144446fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:35.315810 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85ba8a2-3bc0-456f-9b62-a97e144446fc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:35.326975 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1263/2538] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:39:35.331577 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55251a5f-f699-4f5e-8891-26ef6d252aad admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:35.342008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55251a5f-f699-4f5e-8891-26ef6d252aad admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:35.347703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a24756-adf1-4d4a-bb64-86291d0bae4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:35.348423 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a24756-adf1-4d4a-bb64-86291d0bae4c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:35.348713 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a24756-adf1-4d4a-bb64-86291d0bae4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:35.348994 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a24756-adf1-4d4a-bb64-86291d0bae4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:35.349721 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a24756-adf1-4d4a-bb64-86291d0bae4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:35.351348 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a24756-adf1-4d4a-bb64-86291d0bae4c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:35.365602 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1276/2539] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:39:35.369987 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ba8a2-3bc0-456f-9b62-a97e144446fc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:35.387420 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85ba8a2-3bc0-456f-9b62-a97e144446fc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:35.394276 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95474d-2a8a-4600-b29e-36443d6ba677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:35.395752 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95474d-2a8a-4600-b29e-36443d6ba677 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:35.396147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95474d-2a8a-4600-b29e-36443d6ba677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:35.396526 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95474d-2a8a-4600-b29e-36443d6ba677 admin admin] PATH_INFO: `/v3/projects/2887d36c6af144b19c679f193c58da38/users/8a7353fb70e74081ab616917060d4674/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:35.414199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95474d-2a8a-4600-b29e-36443d6ba677 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2887d36c6af144b19c679f193c58da38, user_id=8a7353fb70e74081ab616917060d4674, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:35.417084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95474d-2a8a-4600-b29e-36443d6ba677 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:35.434799 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1264/2540] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:39:35 2022] PUT /identity/v3/projects/2887d36c6af144b19c679f193c58da38/users/8a7353fb70e74081ab616917060d4674/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:39:35.447068 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be17a0-edea-486b-a938-87a2f58429de None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:35.447360 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be17a0-edea-486b-a938-87a2f58429de None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:35.447644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be17a0-edea-486b-a938-87a2f58429de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:35.475840 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b0be17a0-edea-486b-a938-87a2f58429de None None] MFA Rules not processed for user `8a7353fb70e74081ab616917060d4674`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:39:35.477049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b0be17a0-edea-486b-a938-87a2f58429de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:35.552294 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1277/2541] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:39:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:39:35.560072 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95474d-2a8a-4600-b29e-36443d6ba677 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:35.561796 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fe95474d-2a8a-4600-b29e-36443d6ba677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:35.630355 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95474d-2a8a-4600-b29e-36443d6ba677 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:35.634022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7365d2-4de0-4d72-b922-304186c3d0e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:35.634703 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7365d2-4de0-4d72-b922-304186c3d0e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:35.634993 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7365d2-4de0-4d72-b922-304186c3d0e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:35.635271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7365d2-4de0-4d72-b922-304186c3d0e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:35.644844 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7365d2-4de0-4d72-b922-304186c3d0e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:35.646615 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7365d2-4de0-4d72-b922-304186c3d0e1 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:35.656968 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1265/2542] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:39:35 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:39:45.543422 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=79,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:45.652151 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=75,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:47.335249 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be17a0-edea-486b-a938-87a2f58429de None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:47.336641 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b0be17a0-edea-486b-a938-87a2f58429de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:47.417049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be17a0-edea-486b-a938-87a2f58429de None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:47.423041 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d5061-e15a-46e0-9d11-4a917cfdfdc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:47.423620 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d5061-e15a-46e0-9d11-4a917cfdfdc2 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:47.423907 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d5061-e15a-46e0-9d11-4a917cfdfdc2 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:47.424187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d5061-e15a-46e0-9d11-4a917cfdfdc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:47.426130 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-125d5061-e15a-46e0-9d11-4a917cfdfdc2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:47.439106 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d5061-e15a-46e0-9d11-4a917cfdfdc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:47.440685 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d5061-e15a-46e0-9d11-4a917cfdfdc2 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:47.451502 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1278/2543] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:39:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:39:50.082560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d88651-003e-4dbd-b65e-9444e9e89ff6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:50.082560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d88651-003e-4dbd-b65e-9444e9e89ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:50.082560 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d88651-003e-4dbd-b65e-9444e9e89ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:50.100528 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a2d88651-003e-4dbd-b65e-9444e9e89ff6 None None] MFA Rules not processed for user `8c10852a728844c6ae0a48e3772d95b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:39:50.101959 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a2d88651-003e-4dbd-b65e-9444e9e89ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:50.160049 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1266/2544] 10.222.0.52 () {58 vars in 967 bytes} [Wed Jun 29 12:39:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:39:50.172531 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d5061-e15a-46e0-9d11-4a917cfdfdc2 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:50.173959 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-125d5061-e15a-46e0-9d11-4a917cfdfdc2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:50.244142 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d5061-e15a-46e0-9d11-4a917cfdfdc2 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:50.249084 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e96fa06-241d-4536-8e15-ba3deb665c8c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209b0df45f76404da880a9ae0c637d67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:50.249818 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96fa06-241d-4536-8e15-ba3deb665c8c service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:50.250104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96fa06-241d-4536-8e15-ba3deb665c8c service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:50.250382 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e96fa06-241d-4536-8e15-ba3deb665c8c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:50.258249 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e96fa06-241d-4536-8e15-ba3deb665c8c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:50.259974 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e96fa06-241d-4536-8e15-ba3deb665c8c service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:50.268922 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1279/2545] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 12:39:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 12:39:54.167258 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d88651-003e-4dbd-b65e-9444e9e89ff6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:54.230327 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d88651-003e-4dbd-b65e-9444e9e89ff6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:54.235367 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53563d-3ecc-424d-a8f5-c9ad32a9c731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:54.236217 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53563d-3ecc-424d-a8f5-c9ad32a9c731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:54.236497 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53563d-3ecc-424d-a8f5-c9ad32a9c731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:54.236770 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53563d-3ecc-424d-a8f5-c9ad32a9c731 admin admin] PATH_INFO: `/v3/users/8a7353fb70e74081ab616917060d4674` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:54.237927 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53563d-3ecc-424d-a8f5-c9ad32a9c731 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a7353fb70e74081ab616917060d4674)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:54.239880 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53563d-3ecc-424d-a8f5-c9ad32a9c731 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:54.292342 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2b53563d-3ecc-424d-a8f5-c9ad32a9c731 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a7353fb70e74081ab616917060d4674'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:54.300681 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2b53563d-3ecc-424d-a8f5-c9ad32a9c731 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a7353fb70e74081ab616917060d4674'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:54.313227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2b53563d-3ecc-424d-a8f5-c9ad32a9c731 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a7353fb70e74081ab616917060d4674'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:54.313847 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2b53563d-3ecc-424d-a8f5-c9ad32a9c731 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a7353fb70e74081ab616917060d4674'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:54.314561 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2b53563d-3ecc-424d-a8f5-c9ad32a9c731 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a7353fb70e74081ab616917060d4674'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:54.326727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1267/2546] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:39:54 2022] DELETE /identity/v3/users/8a7353fb70e74081ab616917060d4674 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:39:54.332044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb797de-fa97-403f-ae03-5ee9dd625a2a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:54.332570 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb797de-fa97-403f-ae03-5ee9dd625a2a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:54.332691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb797de-fa97-403f-ae03-5ee9dd625a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:54.349420 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4bb797de-fa97-403f-ae03-5ee9dd625a2a None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:39:54.350642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4bb797de-fa97-403f-ae03-5ee9dd625a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:54.411314 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1280/2547] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:39:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:39:54.421419 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53563d-3ecc-424d-a8f5-c9ad32a9c731 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:54.422945 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2b53563d-3ecc-424d-a8f5-c9ad32a9c731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:54.450323 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:54.480979 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53563d-3ecc-424d-a8f5-c9ad32a9c731 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:54.485162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5aed8-6271-4944-aca7-3090eff0e54b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:54.485792 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5aed8-6271-4944-aca7-3090eff0e54b service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:54.486077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5aed8-6271-4944-aca7-3090eff0e54b service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:54.486352 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5aed8-6271-4944-aca7-3090eff0e54b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:54.495539 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5aed8-6271-4944-aca7-3090eff0e54b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:54.496971 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5aed8-6271-4944-aca7-3090eff0e54b service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:54.506386 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1268/2548] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:39:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:39:54.772928 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb797de-fa97-403f-ae03-5ee9dd625a2a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:39:54.774386 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4bb797de-fa97-403f-ae03-5ee9dd625a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:39:54.787115 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb797de-fa97-403f-ae03-5ee9dd625a2a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:39:54.791830 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57b3f1-849a-4b0e-bf0f-e28d07e31b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:39:54.792708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57b3f1-849a-4b0e-bf0f-e28d07e31b0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:39:54.792998 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57b3f1-849a-4b0e-bf0f-e28d07e31b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:39:54.793278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d57b3f1-849a-4b0e-bf0f-e28d07e31b0f admin admin] PATH_INFO: `/v3/projects/2887d36c6af144b19c679f193c58da38` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:39:54.794498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d57b3f1-849a-4b0e-bf0f-e28d07e31b0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2887d36c6af144b19c679f193c58da38)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:39:54.796562 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d57b3f1-849a-4b0e-bf0f-e28d07e31b0f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:39:54.807830 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:39:54.846141 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8d57b3f1-849a-4b0e-bf0f-e28d07e31b0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2887d36c6af144b19c679f193c58da38', 'request_id': 'req-8d57b3f1-849a-4b0e-bf0f-e28d07e31b0f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:54.853561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8d57b3f1-849a-4b0e-bf0f-e28d07e31b0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2887d36c6af144b19c679f193c58da38', 'request_id': 'req-8d57b3f1-849a-4b0e-bf0f-e28d07e31b0f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:54.859792 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8d57b3f1-849a-4b0e-bf0f-e28d07e31b0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2887d36c6af144b19c679f193c58da38', 'request_id': 'req-8d57b3f1-849a-4b0e-bf0f-e28d07e31b0f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:54.869799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8d57b3f1-849a-4b0e-bf0f-e28d07e31b0f admin admin] The token cache is being invalidate because project 2887d36c6af144b19c679f193c58da38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:39:54.870083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8d57b3f1-849a-4b0e-bf0f-e28d07e31b0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:54.870729 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8d57b3f1-849a-4b0e-bf0f-e28d07e31b0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:39:54.872242 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1281/2549] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:39:54 2022] DELETE /identity/v3/projects/2887d36c6af144b19c679f193c58da38 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:40:04.502191 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=15 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:40:04.866858 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=20,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:40:15.283093 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5aed8-6271-4944-aca7-3090eff0e54b service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:15.284495 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c2e5aed8-6271-4944-aca7-3090eff0e54b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:40:15.296120 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5aed8-6271-4944-aca7-3090eff0e54b service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:15.345441 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7604550-fe90-4071-9743-fdbbb7b308ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:15.346013 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7604550-fe90-4071-9743-fdbbb7b308ab service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:15.346306 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7604550-fe90-4071-9743-fdbbb7b308ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:15.346619 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7604550-fe90-4071-9743-fdbbb7b308ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:15.348407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d7604550-fe90-4071-9743-fdbbb7b308ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:40:15.413488 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7604550-fe90-4071-9743-fdbbb7b308ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:15.414974 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7604550-fe90-4071-9743-fdbbb7b308ab service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:15.425553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1269/2550] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:40:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:40:21.632334 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57b3f1-849a-4b0e-bf0f-e28d07e31b0f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:21.633909 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8d57b3f1-849a-4b0e-bf0f-e28d07e31b0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:40:21.689766 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7604550-fe90-4071-9743-fdbbb7b308ab service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:21.691289 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d7604550-fe90-4071-9743-fdbbb7b308ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:40:21.704848 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7604550-fe90-4071-9743-fdbbb7b308ab service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:21.709676 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e293cc3-2bf0-4284-8a1e-263473368027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:21.710582 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e293cc3-2bf0-4284-8a1e-263473368027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:21.710869 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e293cc3-2bf0-4284-8a1e-263473368027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:21.711147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e293cc3-2bf0-4284-8a1e-263473368027 admin admin] PATH_INFO: `/v3/users/f026018d7f444519983df75eb4453284` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:21.712327 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e293cc3-2bf0-4284-8a1e-263473368027 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f026018d7f444519983df75eb4453284)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:21.714631 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e293cc3-2bf0-4284-8a1e-263473368027 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:21.724751 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d57b3f1-849a-4b0e-bf0f-e28d07e31b0f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:21.729727 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd34eb-7684-4fa3-9487-68f01b9cbd18 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98135cacb9d4f22846ad33500b9bd1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:21.730386 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd34eb-7684-4fa3-9487-68f01b9cbd18 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:21.730712 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd34eb-7684-4fa3-9487-68f01b9cbd18 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:21.730999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd34eb-7684-4fa3-9487-68f01b9cbd18 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:21.732849 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-71dd34eb-7684-4fa3-9487-68f01b9cbd18 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:40:21.746599 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd34eb-7684-4fa3-9487-68f01b9cbd18 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:21.748351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd34eb-7684-4fa3-9487-68f01b9cbd18 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:21.759305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1282/2551] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:40:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 12:40:21.762467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9e293cc3-2bf0-4284-8a1e-263473368027 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f026018d7f444519983df75eb4453284'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:21.771337 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9e293cc3-2bf0-4284-8a1e-263473368027 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f026018d7f444519983df75eb4453284'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:21.785036 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9e293cc3-2bf0-4284-8a1e-263473368027 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f026018d7f444519983df75eb4453284'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:21.785726 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9e293cc3-2bf0-4284-8a1e-263473368027 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f026018d7f444519983df75eb4453284'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:21.786372 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9e293cc3-2bf0-4284-8a1e-263473368027 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f026018d7f444519983df75eb4453284'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:21.798975 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1270/2552] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:40:21 2022] DELETE /identity/v3/users/f026018d7f444519983df75eb4453284 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:40:21.972016 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd34eb-7684-4fa3-9487-68f01b9cbd18 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:21.985599 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd34eb-7684-4fa3-9487-68f01b9cbd18 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:22.043608 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f0e3e-7bdb-498b-be78-9f6cc4e930cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:22.044468 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f0e3e-7bdb-498b-be78-9f6cc4e930cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:22.044750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f0e3e-7bdb-498b-be78-9f6cc4e930cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:22.045025 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f0e3e-7bdb-498b-be78-9f6cc4e930cb admin admin] PATH_INFO: `/v3/projects/aacced43dbbb4c2e98bdff0d3cc557ad` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:22.046188 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9f0e3e-7bdb-498b-be78-9f6cc4e930cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=aacced43dbbb4c2e98bdff0d3cc557ad)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:22.048748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9f0e3e-7bdb-498b-be78-9f6cc4e930cb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:22.096780 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7e9f0e3e-7bdb-498b-be78-9f6cc4e930cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aacced43dbbb4c2e98bdff0d3cc557ad', 'request_id': 'req-7e9f0e3e-7bdb-498b-be78-9f6cc4e930cb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:22.104500 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7e9f0e3e-7bdb-498b-be78-9f6cc4e930cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aacced43dbbb4c2e98bdff0d3cc557ad', 'request_id': 'req-7e9f0e3e-7bdb-498b-be78-9f6cc4e930cb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:22.111556 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7e9f0e3e-7bdb-498b-be78-9f6cc4e930cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aacced43dbbb4c2e98bdff0d3cc557ad', 'request_id': 'req-7e9f0e3e-7bdb-498b-be78-9f6cc4e930cb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:22.121971 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7e9f0e3e-7bdb-498b-be78-9f6cc4e930cb admin admin] The token cache is being invalidate because project aacced43dbbb4c2e98bdff0d3cc557ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:40:22.122215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7e9f0e3e-7bdb-498b-be78-9f6cc4e930cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:22.122854 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7e9f0e3e-7bdb-498b-be78-9f6cc4e930cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:22.124414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1283/2553] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:40:21 2022] DELETE /identity/v3/projects/aacced43dbbb4c2e98bdff0d3cc557ad => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:40:22.128492 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e293cc3-2bf0-4284-8a1e-263473368027 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:22.129924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9e293cc3-2bf0-4284-8a1e-263473368027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:40:22.143521 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e293cc3-2bf0-4284-8a1e-263473368027 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:22.162559 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:40:22.203671 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7c3c3-95c5-4b38-be50-371de0c073b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:22.204518 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7c3c3-95c5-4b38-be50-371de0c073b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:22.204801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7c3c3-95c5-4b38-be50-371de0c073b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:22.205078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c7c3c3-95c5-4b38-be50-371de0c073b3 admin admin] PATH_INFO: `/v3/users/5d8b3b503b37428380d7268b4a1e255c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:22.206267 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7c3c3-95c5-4b38-be50-371de0c073b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d8b3b503b37428380d7268b4a1e255c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:22.208235 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c7c3c3-95c5-4b38-be50-371de0c073b3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:22.254493 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-40c7c3c3-95c5-4b38-be50-371de0c073b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d8b3b503b37428380d7268b4a1e255c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:22.261754 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-40c7c3c3-95c5-4b38-be50-371de0c073b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d8b3b503b37428380d7268b4a1e255c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:22.274908 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-40c7c3c3-95c5-4b38-be50-371de0c073b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d8b3b503b37428380d7268b4a1e255c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:22.275551 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-40c7c3c3-95c5-4b38-be50-371de0c073b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d8b3b503b37428380d7268b4a1e255c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:22.276154 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-40c7c3c3-95c5-4b38-be50-371de0c073b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d8b3b503b37428380d7268b4a1e255c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:22.288933 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1271/2554] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:40:22 2022] DELETE /identity/v3/users/5d8b3b503b37428380d7268b4a1e255c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:40:22.473806 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f0e3e-7bdb-498b-be78-9f6cc4e930cb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:22.475487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7e9f0e3e-7bdb-498b-be78-9f6cc4e930cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:40:22.491234 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f0e3e-7bdb-498b-be78-9f6cc4e930cb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:22.511109 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:40:22.552213 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e1641-e5da-48cb-9cfe-5e42522d6ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:22.553077 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e1641-e5da-48cb-9cfe-5e42522d6ecf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:22.553356 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e1641-e5da-48cb-9cfe-5e42522d6ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:22.553629 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e1641-e5da-48cb-9cfe-5e42522d6ecf admin admin] PATH_INFO: `/v3/projects/b499b15ff154498a96a5d304ded4cb8f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:22.554918 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e1641-e5da-48cb-9cfe-5e42522d6ecf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b499b15ff154498a96a5d304ded4cb8f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:22.556925 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e1641-e5da-48cb-9cfe-5e42522d6ecf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:22.603531 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-359e1641-e5da-48cb-9cfe-5e42522d6ecf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b499b15ff154498a96a5d304ded4cb8f', 'request_id': 'req-359e1641-e5da-48cb-9cfe-5e42522d6ecf'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:22.612442 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-359e1641-e5da-48cb-9cfe-5e42522d6ecf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b499b15ff154498a96a5d304ded4cb8f', 'request_id': 'req-359e1641-e5da-48cb-9cfe-5e42522d6ecf'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:22.618888 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-359e1641-e5da-48cb-9cfe-5e42522d6ecf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b499b15ff154498a96a5d304ded4cb8f', 'request_id': 'req-359e1641-e5da-48cb-9cfe-5e42522d6ecf'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:22.630201 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-359e1641-e5da-48cb-9cfe-5e42522d6ecf admin admin] The token cache is being invalidate because project b499b15ff154498a96a5d304ded4cb8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:40:22.630516 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-359e1641-e5da-48cb-9cfe-5e42522d6ecf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:22.631145 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-359e1641-e5da-48cb-9cfe-5e42522d6ecf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:22.632690 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1284/2555] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:40:22 2022] DELETE /identity/v3/projects/b499b15ff154498a96a5d304ded4cb8f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:40:31.771579 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7c3c3-95c5-4b38-be50-371de0c073b3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:31.773109 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-40c7c3c3-95c5-4b38-be50-371de0c073b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:40:31.846928 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c7c3c3-95c5-4b38-be50-371de0c073b3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:31.929680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbeb3b9-87a3-4cbf-9a43-c069d63846c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:31.930523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbeb3b9-87a3-4cbf-9a43-c069d63846c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:31.930845 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbeb3b9-87a3-4cbf-9a43-c069d63846c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:31.931308 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbeb3b9-87a3-4cbf-9a43-c069d63846c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:31.933671 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6fbeb3b9-87a3-4cbf-9a43-c069d63846c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:40:31.946665 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:40:32.008712 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbeb3b9-87a3-4cbf-9a43-c069d63846c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:32.010306 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbeb3b9-87a3-4cbf-9a43-c069d63846c5 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:32.021230 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1272/2556] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:40:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:40:32.627133 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:40:35.209828 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f838c6-8afd-458b-96b8-0486110584e8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:35.210669 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f838c6-8afd-458b-96b8-0486110584e8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:35.210669 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f838c6-8afd-458b-96b8-0486110584e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:35.232411 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-70f838c6-8afd-458b-96b8-0486110584e8 None None] MFA Rules not processed for user `2af4d4a69e9d4755bff61401f4349c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:40:35.232913 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-70f838c6-8afd-458b-96b8-0486110584e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:40:35.328713 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1285/2557] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:40:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:40:35.739280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbeb3b9-87a3-4cbf-9a43-c069d63846c5 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:35.741368 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6fbeb3b9-87a3-4cbf-9a43-c069d63846c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:40:35.758244 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbeb3b9-87a3-4cbf-9a43-c069d63846c5 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:35.763191 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8925f4-bec6-40c8-842d-4c3dcdadc145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:35.764330 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8925f4-bec6-40c8-842d-4c3dcdadc145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:35.764614 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8925f4-bec6-40c8-842d-4c3dcdadc145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:35.764889 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8925f4-bec6-40c8-842d-4c3dcdadc145 admin admin] PATH_INFO: `/v3/users/6252d0519ed9421e89710ca5294524c5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:35.766273 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8925f4-bec6-40c8-842d-4c3dcdadc145 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6252d0519ed9421e89710ca5294524c5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:35.769094 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8925f4-bec6-40c8-842d-4c3dcdadc145 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:35.823987 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ce8925f4-bec6-40c8-842d-4c3dcdadc145 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6252d0519ed9421e89710ca5294524c5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:35.835720 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ce8925f4-bec6-40c8-842d-4c3dcdadc145 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6252d0519ed9421e89710ca5294524c5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:35.849727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ce8925f4-bec6-40c8-842d-4c3dcdadc145 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6252d0519ed9421e89710ca5294524c5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:35.850684 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ce8925f4-bec6-40c8-842d-4c3dcdadc145 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6252d0519ed9421e89710ca5294524c5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:35.851450 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ce8925f4-bec6-40c8-842d-4c3dcdadc145 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6252d0519ed9421e89710ca5294524c5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:35.867356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1273/2558] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:40:35 2022] DELETE /identity/v3/users/6252d0519ed9421e89710ca5294524c5 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:40:36.089654 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f838c6-8afd-458b-96b8-0486110584e8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:36.103134 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f838c6-8afd-458b-96b8-0486110584e8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:36.163322 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750af8d9-16df-4752-8f54-d47562d2cbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:36.164188 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750af8d9-16df-4752-8f54-d47562d2cbad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:36.164541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750af8d9-16df-4752-8f54-d47562d2cbad admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:36.164830 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750af8d9-16df-4752-8f54-d47562d2cbad admin admin] PATH_INFO: `/v3/projects/0ed882ae485441db99ccf6109be51b22` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:36.166173 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750af8d9-16df-4752-8f54-d47562d2cbad admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ed882ae485441db99ccf6109be51b22)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:36.168445 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750af8d9-16df-4752-8f54-d47562d2cbad admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:36.219439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-750af8d9-16df-4752-8f54-d47562d2cbad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ed882ae485441db99ccf6109be51b22', 'request_id': 'req-750af8d9-16df-4752-8f54-d47562d2cbad'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:36.226976 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-750af8d9-16df-4752-8f54-d47562d2cbad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ed882ae485441db99ccf6109be51b22', 'request_id': 'req-750af8d9-16df-4752-8f54-d47562d2cbad'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:36.233330 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-750af8d9-16df-4752-8f54-d47562d2cbad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ed882ae485441db99ccf6109be51b22', 'request_id': 'req-750af8d9-16df-4752-8f54-d47562d2cbad'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:36.244780 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-750af8d9-16df-4752-8f54-d47562d2cbad admin admin] The token cache is being invalidate because project 0ed882ae485441db99ccf6109be51b22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:40:36.245156 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-750af8d9-16df-4752-8f54-d47562d2cbad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:36.245906 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-750af8d9-16df-4752-8f54-d47562d2cbad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:36.252113 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1286/2559] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:40:36 2022] DELETE /identity/v3/projects/0ed882ae485441db99ccf6109be51b22 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:40:36.254092 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4153e-ff7a-481d-aa36-5f44e9a5a3ad None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:36.254382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4153e-ff7a-481d-aa36-5f44e9a5a3ad None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:36.254693 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4153e-ff7a-481d-aa36-5f44e9a5a3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:36.274700 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2ea4153e-ff7a-481d-aa36-5f44e9a5a3ad None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:40:36.275918 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2ea4153e-ff7a-481d-aa36-5f44e9a5a3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:40:36.339339 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1274/2560] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:40:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:40:36.347755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7adc0-a000-4f94-99c8-05a8ec2d6db9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:36.348062 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7adc0-a000-4f94-99c8-05a8ec2d6db9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:36.348391 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab7adc0-a000-4f94-99c8-05a8ec2d6db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:36.369211 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2ab7adc0-a000-4f94-99c8-05a8ec2d6db9 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:40:36.370539 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2ab7adc0-a000-4f94-99c8-05a8ec2d6db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:40:36.379186 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1287/2561] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:40:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:40:36.383420 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4153e-ff7a-481d-aa36-5f44e9a5a3ad None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:36.394169 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4153e-ff7a-481d-aa36-5f44e9a5a3ad None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:36.399168 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355a9dc-8ad3-4051-be30-896104dec2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:36.399821 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355a9dc-8ad3-4051-be30-896104dec2ca admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:36.400103 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355a9dc-8ad3-4051-be30-896104dec2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:36.400378 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355a9dc-8ad3-4051-be30-896104dec2ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:36.401194 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355a9dc-8ad3-4051-be30-896104dec2ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:40:36.401508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355a9dc-8ad3-4051-be30-896104dec2ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:36.403160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355a9dc-8ad3-4051-be30-896104dec2ca admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:36.415298 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1275/2562] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:40:36.419526 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7adc0-a000-4f94-99c8-05a8ec2d6db9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:36.433274 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab7adc0-a000-4f94-99c8-05a8ec2d6db9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:36.438833 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982089d-76c4-448c-aec4-70a0bd235cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:36.439502 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982089d-76c4-448c-aec4-70a0bd235cb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:36.439816 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982089d-76c4-448c-aec4-70a0bd235cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:36.440092 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982089d-76c4-448c-aec4-70a0bd235cb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:36.440804 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982089d-76c4-448c-aec4-70a0bd235cb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:36.442945 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982089d-76c4-448c-aec4-70a0bd235cb9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:36.473088 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1288/2563] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:40:36 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:40:36.477338 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355a9dc-8ad3-4051-be30-896104dec2ca admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:36.487823 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355a9dc-8ad3-4051-be30-896104dec2ca admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:36.491645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:40:36.548853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3865bbd-002e-4831-9158-b110f02befcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:36.549441 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3865bbd-002e-4831-9158-b110f02befcb admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:36.549728 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3865bbd-002e-4831-9158-b110f02befcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:36.550061 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3865bbd-002e-4831-9158-b110f02befcb admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:36.550953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3865bbd-002e-4831-9158-b110f02befcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:36.552898 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3865bbd-002e-4831-9158-b110f02befcb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:36.574343 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1276/2564] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:40:36 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:40:36.578259 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982089d-76c4-448c-aec4-70a0bd235cb9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:36.589445 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982089d-76c4-448c-aec4-70a0bd235cb9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:36.594974 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a97c54-06e0-4edc-b683-1e8430e2c9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:36.595649 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a97c54-06e0-4edc-b683-1e8430e2c9ff admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:36.595937 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a97c54-06e0-4edc-b683-1e8430e2c9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:36.596218 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a97c54-06e0-4edc-b683-1e8430e2c9ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:36.596950 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a97c54-06e0-4edc-b683-1e8430e2c9ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:36.598584 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a97c54-06e0-4edc-b683-1e8430e2c9ff admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:36.600047 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:40:36.611312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1289/2565] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:40:36.615418 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3865bbd-002e-4831-9158-b110f02befcb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:36.625603 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3865bbd-002e-4831-9158-b110f02befcb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:36.630136 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ff185-a515-4756-9872-d3c315fd83b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:36.630820 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ff185-a515-4756-9872-d3c315fd83b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:36.631107 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ff185-a515-4756-9872-d3c315fd83b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:36.631386 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ff185-a515-4756-9872-d3c315fd83b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:36.632127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ff185-a515-4756-9872-d3c315fd83b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:36.633723 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7ff185-a515-4756-9872-d3c315fd83b7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:36.643895 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1277/2566] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:40:36.648202 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a97c54-06e0-4edc-b683-1e8430e2c9ff admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:36.658712 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a97c54-06e0-4edc-b683-1e8430e2c9ff admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:36.663325 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916184c9-9c26-44cb-9531-6047c50767c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:36.664198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916184c9-9c26-44cb-9531-6047c50767c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:36.664483 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916184c9-9c26-44cb-9531-6047c50767c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:36.664761 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916184c9-9c26-44cb-9531-6047c50767c8 admin admin] PATH_INFO: `/v3/projects/a485b1538d984e1680e8360a5060bd8e/users/1eff347abf6b4362971126adb984ad16/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:36.680768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916184c9-9c26-44cb-9531-6047c50767c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a485b1538d984e1680e8360a5060bd8e, user_id=1eff347abf6b4362971126adb984ad16, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:36.683464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916184c9-9c26-44cb-9531-6047c50767c8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:36.697355 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1290/2567] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:40:36 2022] PUT /identity/v3/projects/a485b1538d984e1680e8360a5060bd8e/users/1eff347abf6b4362971126adb984ad16/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:40:36.708221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df81d84-de98-4a08-9d5e-425382f04027 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:36.708569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df81d84-de98-4a08-9d5e-425382f04027 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:36.708853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df81d84-de98-4a08-9d5e-425382f04027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:36.736312 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9df81d84-de98-4a08-9d5e-425382f04027 None None] MFA Rules not processed for user `1eff347abf6b4362971126adb984ad16`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:40:36.737528 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9df81d84-de98-4a08-9d5e-425382f04027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:40:36.803226 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1278/2568] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:40:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:40:36.814285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916184c9-9c26-44cb-9531-6047c50767c8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:36.816190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-916184c9-9c26-44cb-9531-6047c50767c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:40:36.899372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916184c9-9c26-44cb-9531-6047c50767c8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:36.905296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2802e9d-5ec1-4d16-a4a5-1a478d28fee5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:36.906557 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2802e9d-5ec1-4d16-a4a5-1a478d28fee5 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:36.906843 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2802e9d-5ec1-4d16-a4a5-1a478d28fee5 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:36.907119 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2802e9d-5ec1-4d16-a4a5-1a478d28fee5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:36.916375 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2802e9d-5ec1-4d16-a4a5-1a478d28fee5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:36.917990 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2802e9d-5ec1-4d16-a4a5-1a478d28fee5 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:36.928414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1291/2569] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:40:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:40:44.313761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df81d84-de98-4a08-9d5e-425382f04027 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:44.376826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df81d84-de98-4a08-9d5e-425382f04027 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:44.381436 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c02d384-203c-45ab-a55b-bfc8bf8c08ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:44.382323 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02d384-203c-45ab-a55b-bfc8bf8c08ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:44.382650 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02d384-203c-45ab-a55b-bfc8bf8c08ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:44.382934 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02d384-203c-45ab-a55b-bfc8bf8c08ba admin admin] PATH_INFO: `/v3/users/1eff347abf6b4362971126adb984ad16` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:44.384132 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c02d384-203c-45ab-a55b-bfc8bf8c08ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=1eff347abf6b4362971126adb984ad16)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:44.386056 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c02d384-203c-45ab-a55b-bfc8bf8c08ba admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:44.401806 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:40:44.431874 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5c02d384-203c-45ab-a55b-bfc8bf8c08ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1eff347abf6b4362971126adb984ad16'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:44.440283 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5c02d384-203c-45ab-a55b-bfc8bf8c08ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1eff347abf6b4362971126adb984ad16'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:44.452390 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5c02d384-203c-45ab-a55b-bfc8bf8c08ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1eff347abf6b4362971126adb984ad16'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:44.453058 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5c02d384-203c-45ab-a55b-bfc8bf8c08ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1eff347abf6b4362971126adb984ad16'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:44.453667 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5c02d384-203c-45ab-a55b-bfc8bf8c08ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1eff347abf6b4362971126adb984ad16'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:44.465850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1279/2570] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:40:44 2022] DELETE /identity/v3/users/1eff347abf6b4362971126adb984ad16 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:40:44.474381 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72206ac0-a7d9-4bfe-be46-3b1a61b93c73 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:44.474963 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72206ac0-a7d9-4bfe-be46-3b1a61b93c73 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:44.474963 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72206ac0-a7d9-4bfe-be46-3b1a61b93c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:44.493912 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-72206ac0-a7d9-4bfe-be46-3b1a61b93c73 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:40:44.495164 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-72206ac0-a7d9-4bfe-be46-3b1a61b93c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:40:44.556692 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1292/2571] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:40:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:40:44.566727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c02d384-203c-45ab-a55b-bfc8bf8c08ba admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:44.568112 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5c02d384-203c-45ab-a55b-bfc8bf8c08ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:40:44.624175 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c02d384-203c-45ab-a55b-bfc8bf8c08ba admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:44.627743 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f613b3-e184-429c-9d14-aac9f0b6c07a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:44.628379 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f613b3-e184-429c-9d14-aac9f0b6c07a service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:44.628668 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f613b3-e184-429c-9d14-aac9f0b6c07a service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:44.628945 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f613b3-e184-429c-9d14-aac9f0b6c07a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:44.637749 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f613b3-e184-429c-9d14-aac9f0b6c07a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:44.639211 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f613b3-e184-429c-9d14-aac9f0b6c07a service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:44.648888 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1280/2572] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:40:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:40:44.915602 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72206ac0-a7d9-4bfe-be46-3b1a61b93c73 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:44.917299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-72206ac0-a7d9-4bfe-be46-3b1a61b93c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:40:44.933133 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72206ac0-a7d9-4bfe-be46-3b1a61b93c73 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:44.938722 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe25ab4-1d8c-4d1c-b93f-6e971f2ee503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:44.939597 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe25ab4-1d8c-4d1c-b93f-6e971f2ee503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:44.939880 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe25ab4-1d8c-4d1c-b93f-6e971f2ee503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:44.940155 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe25ab4-1d8c-4d1c-b93f-6e971f2ee503 admin admin] PATH_INFO: `/v3/projects/a485b1538d984e1680e8360a5060bd8e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:44.941363 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe25ab4-1d8c-4d1c-b93f-6e971f2ee503 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a485b1538d984e1680e8360a5060bd8e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:44.943387 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe25ab4-1d8c-4d1c-b93f-6e971f2ee503 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:44.993879 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3fe25ab4-1d8c-4d1c-b93f-6e971f2ee503 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a485b1538d984e1680e8360a5060bd8e', 'request_id': 'req-3fe25ab4-1d8c-4d1c-b93f-6e971f2ee503'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:44.997131 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:40:45.004033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3fe25ab4-1d8c-4d1c-b93f-6e971f2ee503 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a485b1538d984e1680e8360a5060bd8e', 'request_id': 'req-3fe25ab4-1d8c-4d1c-b93f-6e971f2ee503'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:45.012354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3fe25ab4-1d8c-4d1c-b93f-6e971f2ee503 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a485b1538d984e1680e8360a5060bd8e', 'request_id': 'req-3fe25ab4-1d8c-4d1c-b93f-6e971f2ee503'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:45.024117 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3fe25ab4-1d8c-4d1c-b93f-6e971f2ee503 admin admin] The token cache is being invalidate because project a485b1538d984e1680e8360a5060bd8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:40:45.024410 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3fe25ab4-1d8c-4d1c-b93f-6e971f2ee503 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:45.025097 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3fe25ab4-1d8c-4d1c-b93f-6e971f2ee503 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:40:45.026924 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1293/2573] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:40:44 2022] DELETE /identity/v3/projects/a485b1538d984e1680e8360a5060bd8e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:40:45.032266 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadeb314-acca-4c0e-b2d7-d87a2444f4b9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:45.032559 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadeb314-acca-4c0e-b2d7-d87a2444f4b9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:45.032842 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadeb314-acca-4c0e-b2d7-d87a2444f4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:45.050317 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-aadeb314-acca-4c0e-b2d7-d87a2444f4b9 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:40:45.051573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-aadeb314-acca-4c0e-b2d7-d87a2444f4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:40:45.117692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1281/2574] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:40:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:40:45.126637 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a614c7-87c2-43c4-82a6-3a6a824ae1f2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:45.128526 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a614c7-87c2-43c4-82a6-3a6a824ae1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:45.129121 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a614c7-87c2-43c4-82a6-3a6a824ae1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:45.149608 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f1a614c7-87c2-43c4-82a6-3a6a824ae1f2 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:40:45.150861 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f1a614c7-87c2-43c4-82a6-3a6a824ae1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:40:45.159974 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1294/2575] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:40:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:40:45.164297 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadeb314-acca-4c0e-b2d7-d87a2444f4b9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:45.174916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadeb314-acca-4c0e-b2d7-d87a2444f4b9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:45.179657 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0cd8d9-717c-4e3d-a8cc-a1676a577462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:45.180322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cd8d9-717c-4e3d-a8cc-a1676a577462 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:45.180609 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cd8d9-717c-4e3d-a8cc-a1676a577462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:45.181204 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cd8d9-717c-4e3d-a8cc-a1676a577462 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:45.181966 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0cd8d9-717c-4e3d-a8cc-a1676a577462 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:40:45.182277 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0cd8d9-717c-4e3d-a8cc-a1676a577462 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:45.183920 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0cd8d9-717c-4e3d-a8cc-a1676a577462 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:45.198077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1282/2576] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:40:45.203478 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a614c7-87c2-43c4-82a6-3a6a824ae1f2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:45.214115 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a614c7-87c2-43c4-82a6-3a6a824ae1f2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:45.221894 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ff175-cfda-422e-a63d-dc65eef0b024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:45.222736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ff175-cfda-422e-a63d-dc65eef0b024 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:45.223033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ff175-cfda-422e-a63d-dc65eef0b024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:45.223314 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ff175-cfda-422e-a63d-dc65eef0b024 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:45.224047 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39ff175-cfda-422e-a63d-dc65eef0b024 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:45.225991 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39ff175-cfda-422e-a63d-dc65eef0b024 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:45.261563 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1295/2577] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:40:45 2022] POST /identity/v3/projects => generated 355 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:40:45.265670 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0cd8d9-717c-4e3d-a8cc-a1676a577462 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:45.276605 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0cd8d9-717c-4e3d-a8cc-a1676a577462 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:45.302690 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:40:45.351055 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571da60e-d141-4fa9-a041-23640ebb5870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:45.351900 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571da60e-d141-4fa9-a041-23640ebb5870 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:45.352287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571da60e-d141-4fa9-a041-23640ebb5870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:45.352821 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571da60e-d141-4fa9-a041-23640ebb5870 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:45.353504 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571da60e-d141-4fa9-a041-23640ebb5870 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:45.355570 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571da60e-d141-4fa9-a041-23640ebb5870 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:45.382379 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1283/2578] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:40:45 2022] POST /identity/v3/users => generated 278 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:40:45.383625 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ff175-cfda-422e-a63d-dc65eef0b024 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:45.396526 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ff175-cfda-422e-a63d-dc65eef0b024 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:45.401411 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964b83e-d022-4368-861c-29d4006c9a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:45.402195 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964b83e-d022-4368-861c-29d4006c9a00 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:45.402365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964b83e-d022-4368-861c-29d4006c9a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:45.402663 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964b83e-d022-4368-861c-29d4006c9a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:45.403390 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a964b83e-d022-4368-861c-29d4006c9a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:45.405072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a964b83e-d022-4368-861c-29d4006c9a00 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:45.420298 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1296/2579] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:40:45.424783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571da60e-d141-4fa9-a041-23640ebb5870 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:45.435226 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571da60e-d141-4fa9-a041-23640ebb5870 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:45.440357 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c4fa6-df12-4eed-a2a7-6c72346bd326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:45.441027 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c4fa6-df12-4eed-a2a7-6c72346bd326 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:45.441319 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c4fa6-df12-4eed-a2a7-6c72346bd326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:45.441603 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c4fa6-df12-4eed-a2a7-6c72346bd326 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:45.442328 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449c4fa6-df12-4eed-a2a7-6c72346bd326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:45.443981 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449c4fa6-df12-4eed-a2a7-6c72346bd326 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:45.458595 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1284/2580] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:40:45.459938 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964b83e-d022-4368-861c-29d4006c9a00 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:45.471236 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964b83e-d022-4368-861c-29d4006c9a00 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:45.475871 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595409d7-635c-4144-b27f-ef482889d82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:45.476776 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595409d7-635c-4144-b27f-ef482889d82e admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:45.477172 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595409d7-635c-4144-b27f-ef482889d82e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:45.477450 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595409d7-635c-4144-b27f-ef482889d82e admin admin] PATH_INFO: `/v3/projects/f17e1d3b189c43bdb0acdefc0393660f/users/0954f7eee2454dcb91c6fe394359fa26/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:45.494502 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595409d7-635c-4144-b27f-ef482889d82e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f17e1d3b189c43bdb0acdefc0393660f, user_id=0954f7eee2454dcb91c6fe394359fa26, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:45.497348 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595409d7-635c-4144-b27f-ef482889d82e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:45.512152 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1297/2581] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:40:45 2022] PUT /identity/v3/projects/f17e1d3b189c43bdb0acdefc0393660f/users/0954f7eee2454dcb91c6fe394359fa26/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:40:45.523210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d1f75-1063-4ad8-935e-e120dab92d37 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:45.523591 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d1f75-1063-4ad8-935e-e120dab92d37 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:45.523940 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d1f75-1063-4ad8-935e-e120dab92d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:45.551396 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-299d1f75-1063-4ad8-935e-e120dab92d37 None None] MFA Rules not processed for user `0954f7eee2454dcb91c6fe394359fa26`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:40:45.552677 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-299d1f75-1063-4ad8-935e-e120dab92d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:40:45.617053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1285/2582] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:40:45 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:40:45.621490 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595409d7-635c-4144-b27f-ef482889d82e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:45.684928 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595409d7-635c-4144-b27f-ef482889d82e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:45.690045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfaad13-4c91-4cf6-b9f2-848016a97b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:45.690804 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfaad13-4c91-4cf6-b9f2-848016a97b48 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:45.691091 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfaad13-4c91-4cf6-b9f2-848016a97b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:45.691365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfaad13-4c91-4cf6-b9f2-848016a97b48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:45.692057 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfaad13-4c91-4cf6-b9f2-848016a97b48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:45.693924 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfaad13-4c91-4cf6-b9f2-848016a97b48 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:45.723248 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1298/2583] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:40:45 2022] POST /identity/v3/projects => generated 355 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:40:45.727344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d1f75-1063-4ad8-935e-e120dab92d37 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:45.738681 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299d1f75-1063-4ad8-935e-e120dab92d37 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:45.796750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46174a2-a21c-44d0-ac29-ab1985dde2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:45.797424 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46174a2-a21c-44d0-ac29-ab1985dde2ce admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:45.797705 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46174a2-a21c-44d0-ac29-ab1985dde2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:45.797977 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46174a2-a21c-44d0-ac29-ab1985dde2ce admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:45.798704 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46174a2-a21c-44d0-ac29-ab1985dde2ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:45.800610 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46174a2-a21c-44d0-ac29-ab1985dde2ce admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:45.821514 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1286/2584] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:40:45 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:40:45.825470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfaad13-4c91-4cf6-b9f2-848016a97b48 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:45.836149 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfaad13-4c91-4cf6-b9f2-848016a97b48 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:45.840755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfae8db-f8b4-429a-80c5-ca26651afe82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:45.841413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfae8db-f8b4-429a-80c5-ca26651afe82 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:45.841704 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfae8db-f8b4-429a-80c5-ca26651afe82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:45.841986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfae8db-f8b4-429a-80c5-ca26651afe82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:45.842745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfae8db-f8b4-429a-80c5-ca26651afe82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:45.844675 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfae8db-f8b4-429a-80c5-ca26651afe82 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:45.855089 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1299/2585] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:40:45.860105 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46174a2-a21c-44d0-ac29-ab1985dde2ce admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:45.872643 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46174a2-a21c-44d0-ac29-ab1985dde2ce admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:45.878639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ab872-75f4-4aa1-9700-7a2e4c595b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:45.879524 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ab872-75f4-4aa1-9700-7a2e4c595b9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:45.879810 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ab872-75f4-4aa1-9700-7a2e4c595b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:45.880092 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ab872-75f4-4aa1-9700-7a2e4c595b9c admin admin] PATH_INFO: `/v3/projects/b2a72345b9724a36a60a5c3c53bf7e56/users/7aae57dc20fa4c5f985300878ad9971c/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:45.895131 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ab872-75f4-4aa1-9700-7a2e4c595b9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2a72345b9724a36a60a5c3c53bf7e56, user_id=7aae57dc20fa4c5f985300878ad9971c, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:45.898498 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ab872-75f4-4aa1-9700-7a2e4c595b9c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:45.907547 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:40:45.914652 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1287/2586] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:40:45 2022] PUT /identity/v3/projects/b2a72345b9724a36a60a5c3c53bf7e56/users/7aae57dc20fa4c5f985300878ad9971c/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:40:45.918626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfae8db-f8b4-429a-80c5-ca26651afe82 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:45.929665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfae8db-f8b4-429a-80c5-ca26651afe82 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:45.975596 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:40:45.987250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7786e43c-df3f-470e-8282-efce363b88a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:45.987910 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786e43c-df3f-470e-8282-efce363b88a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:45.988194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786e43c-df3f-470e-8282-efce363b88a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:45.988474 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7786e43c-df3f-470e-8282-efce363b88a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:45.989188 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7786e43c-df3f-470e-8282-efce363b88a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:45.990801 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7786e43c-df3f-470e-8282-efce363b88a3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:46.001007 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1300/2587] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:40:46.005112 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ab872-75f4-4aa1-9700-7a2e4c595b9c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:46.015355 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ab872-75f4-4aa1-9700-7a2e4c595b9c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:46.019869 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02469af-8fcc-4a70-9523-4992e36cf0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:46.021015 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02469af-8fcc-4a70-9523-4992e36cf0f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:46.021284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02469af-8fcc-4a70-9523-4992e36cf0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:46.021561 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02469af-8fcc-4a70-9523-4992e36cf0f3 admin admin] PATH_INFO: `/v3/projects/b2a72345b9724a36a60a5c3c53bf7e56/users/7aae57dc20fa4c5f985300878ad9971c/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:46.023527 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02469af-8fcc-4a70-9523-4992e36cf0f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2a72345b9724a36a60a5c3c53bf7e56, user_id=7aae57dc20fa4c5f985300878ad9971c, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:46.026150 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02469af-8fcc-4a70-9523-4992e36cf0f3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:46.036440 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1288/2588] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:40:46 2022] PUT /identity/v3/projects/b2a72345b9724a36a60a5c3c53bf7e56/users/7aae57dc20fa4c5f985300878ad9971c/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:40:46.048370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435ec2e-3109-4a59-a4b6-01e804b454f1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:46.048664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435ec2e-3109-4a59-a4b6-01e804b454f1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:46.048946 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f435ec2e-3109-4a59-a4b6-01e804b454f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:46.075174 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f435ec2e-3109-4a59-a4b6-01e804b454f1 None None] MFA Rules not processed for user `7aae57dc20fa4c5f985300878ad9971c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:40:46.076387 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f435ec2e-3109-4a59-a4b6-01e804b454f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:40:46.142981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1301/2589] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:40:46 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:40:46.154223 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02469af-8fcc-4a70-9523-4992e36cf0f3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:46.156649 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f02469af-8fcc-4a70-9523-4992e36cf0f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:40:46.212950 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02469af-8fcc-4a70-9523-4992e36cf0f3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:46.216715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9f471-99c2-489b-a32f-efbd10d53636 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:46.217366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9f471-99c2-489b-a32f-efbd10d53636 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:46.217656 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9f471-99c2-489b-a32f-efbd10d53636 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:46.218587 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9f471-99c2-489b-a32f-efbd10d53636 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:46.229831 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f9f471-99c2-489b-a32f-efbd10d53636 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:46.231249 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f9f471-99c2-489b-a32f-efbd10d53636 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:46.241245 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1289/2590] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:40:46 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:40:46.318858 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435ec2e-3109-4a59-a4b6-01e804b454f1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:40:46.329360 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f435ec2e-3109-4a59-a4b6-01e804b454f1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:40:46.334591 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3385c0-68bf-4823-ae49-7d91648d5964 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:40:46.335957 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3385c0-68bf-4823-ae49-7d91648d5964 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:40:46.336254 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3385c0-68bf-4823-ae49-7d91648d5964 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:40:46.336537 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3385c0-68bf-4823-ae49-7d91648d5964 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:40:46.396011 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3385c0-68bf-4823-ae49-7d91648d5964 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:40:46.397435 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3385c0-68bf-4823-ae49-7d91648d5964 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:40:46.406312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1302/2591] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:40:46 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:40:56.237133 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=32,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:40:56.402192 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=68 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:01.131841 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9f471-99c2-489b-a32f-efbd10d53636 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:01.194011 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9f471-99c2-489b-a32f-efbd10d53636 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:01.198636 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ac0aa-075f-45d6-834f-774725bc66b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:01.199533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ac0aa-075f-45d6-834f-774725bc66b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:01.199820 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ac0aa-075f-45d6-834f-774725bc66b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:01.200106 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ac0aa-075f-45d6-834f-774725bc66b1 admin admin] PATH_INFO: `/v3/users/0954f7eee2454dcb91c6fe394359fa26` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:01.201293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ac0aa-075f-45d6-834f-774725bc66b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0954f7eee2454dcb91c6fe394359fa26)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:01.203941 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ac0aa-075f-45d6-834f-774725bc66b1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:01.249589 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0e8ac0aa-075f-45d6-834f-774725bc66b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0954f7eee2454dcb91c6fe394359fa26'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:01.256109 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0e8ac0aa-075f-45d6-834f-774725bc66b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0954f7eee2454dcb91c6fe394359fa26'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:01.268265 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0e8ac0aa-075f-45d6-834f-774725bc66b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0954f7eee2454dcb91c6fe394359fa26'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:01.268885 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0e8ac0aa-075f-45d6-834f-774725bc66b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0954f7eee2454dcb91c6fe394359fa26'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:01.269478 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0e8ac0aa-075f-45d6-834f-774725bc66b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0954f7eee2454dcb91c6fe394359fa26'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:01.282298 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1290/2592] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:41:01 2022] DELETE /identity/v3/users/0954f7eee2454dcb91c6fe394359fa26 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:01.286773 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65758a4-e37e-4f1a-98b9-7c828c7a5915 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:01.287235 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65758a4-e37e-4f1a-98b9-7c828c7a5915 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:01.287366 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65758a4-e37e-4f1a-98b9-7c828c7a5915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:01.305018 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c65758a4-e37e-4f1a-98b9-7c828c7a5915 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:41:01.306149 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c65758a4-e37e-4f1a-98b9-7c828c7a5915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:01.366343 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1303/2593] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:41:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:41:01.375948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ac0aa-075f-45d6-834f-774725bc66b1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:01.377369 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0e8ac0aa-075f-45d6-834f-774725bc66b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:01.433999 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ac0aa-075f-45d6-834f-774725bc66b1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:01.438188 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff20044-13ac-400f-b817-588446c12a59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:01.438873 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff20044-13ac-400f-b817-588446c12a59 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:01.439160 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff20044-13ac-400f-b817-588446c12a59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:01.439442 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff20044-13ac-400f-b817-588446c12a59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:01.448499 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff20044-13ac-400f-b817-588446c12a59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:01.449929 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff20044-13ac-400f-b817-588446c12a59 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:01.460133 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1291/2594] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:41:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:41:01.720198 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65758a4-e37e-4f1a-98b9-7c828c7a5915 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:01.721647 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c65758a4-e37e-4f1a-98b9-7c828c7a5915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:01.733920 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65758a4-e37e-4f1a-98b9-7c828c7a5915 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:01.738834 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aecc6a-f44b-456d-a1fe-6bfcc5feed0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:01.739704 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aecc6a-f44b-456d-a1fe-6bfcc5feed0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:01.739985 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aecc6a-f44b-456d-a1fe-6bfcc5feed0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:01.740261 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aecc6a-f44b-456d-a1fe-6bfcc5feed0e admin admin] PATH_INFO: `/v3/projects/f17e1d3b189c43bdb0acdefc0393660f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:01.742042 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aecc6a-f44b-456d-a1fe-6bfcc5feed0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f17e1d3b189c43bdb0acdefc0393660f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:01.744187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aecc6a-f44b-456d-a1fe-6bfcc5feed0e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:01.791264 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-38aecc6a-f44b-456d-a1fe-6bfcc5feed0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f17e1d3b189c43bdb0acdefc0393660f', 'request_id': 'req-38aecc6a-f44b-456d-a1fe-6bfcc5feed0e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:01.798431 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-38aecc6a-f44b-456d-a1fe-6bfcc5feed0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f17e1d3b189c43bdb0acdefc0393660f', 'request_id': 'req-38aecc6a-f44b-456d-a1fe-6bfcc5feed0e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:01.804568 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-38aecc6a-f44b-456d-a1fe-6bfcc5feed0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f17e1d3b189c43bdb0acdefc0393660f', 'request_id': 'req-38aecc6a-f44b-456d-a1fe-6bfcc5feed0e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:01.816550 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-38aecc6a-f44b-456d-a1fe-6bfcc5feed0e admin admin] The token cache is being invalidate because project f17e1d3b189c43bdb0acdefc0393660f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:41:01.816815 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-38aecc6a-f44b-456d-a1fe-6bfcc5feed0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:01.817446 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-38aecc6a-f44b-456d-a1fe-6bfcc5feed0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:01.819002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1304/2595] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:41:01 2022] DELETE /identity/v3/projects/f17e1d3b189c43bdb0acdefc0393660f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:41:01.822966 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff20044-13ac-400f-b817-588446c12a59 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:01.833254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff20044-13ac-400f-b817-588446c12a59 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:01.851232 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:01.891096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614cfa0-fed9-4c54-bd53-18a3b1d37b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:01.891864 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614cfa0-fed9-4c54-bd53-18a3b1d37b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:01.892153 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614cfa0-fed9-4c54-bd53-18a3b1d37b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:01.892438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614cfa0-fed9-4c54-bd53-18a3b1d37b79 admin admin] PATH_INFO: `/v3/users/7aae57dc20fa4c5f985300878ad9971c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:01.893644 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3614cfa0-fed9-4c54-bd53-18a3b1d37b79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7aae57dc20fa4c5f985300878ad9971c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:01.895854 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3614cfa0-fed9-4c54-bd53-18a3b1d37b79 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:01.942701 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3614cfa0-fed9-4c54-bd53-18a3b1d37b79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7aae57dc20fa4c5f985300878ad9971c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:01.949989 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3614cfa0-fed9-4c54-bd53-18a3b1d37b79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7aae57dc20fa4c5f985300878ad9971c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:01.964274 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3614cfa0-fed9-4c54-bd53-18a3b1d37b79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7aae57dc20fa4c5f985300878ad9971c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:01.964938 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3614cfa0-fed9-4c54-bd53-18a3b1d37b79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7aae57dc20fa4c5f985300878ad9971c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:01.965548 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3614cfa0-fed9-4c54-bd53-18a3b1d37b79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7aae57dc20fa4c5f985300878ad9971c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:01.978317 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1292/2596] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:41:01 2022] DELETE /identity/v3/users/7aae57dc20fa4c5f985300878ad9971c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:02.258239 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aecc6a-f44b-456d-a1fe-6bfcc5feed0e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:02.259815 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-38aecc6a-f44b-456d-a1fe-6bfcc5feed0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:02.274309 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aecc6a-f44b-456d-a1fe-6bfcc5feed0e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:02.332784 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2637c56-f8fe-48ae-8368-ba2a5576d6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:02.333640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2637c56-f8fe-48ae-8368-ba2a5576d6af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:02.333924 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2637c56-f8fe-48ae-8368-ba2a5576d6af admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:02.334200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2637c56-f8fe-48ae-8368-ba2a5576d6af admin admin] PATH_INFO: `/v3/projects/b2a72345b9724a36a60a5c3c53bf7e56` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:02.335405 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2637c56-f8fe-48ae-8368-ba2a5576d6af admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2a72345b9724a36a60a5c3c53bf7e56)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:02.337326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2637c56-f8fe-48ae-8368-ba2a5576d6af admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:02.368190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:02.386228 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e2637c56-f8fe-48ae-8368-ba2a5576d6af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2a72345b9724a36a60a5c3c53bf7e56', 'request_id': 'req-e2637c56-f8fe-48ae-8368-ba2a5576d6af'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:02.394317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e2637c56-f8fe-48ae-8368-ba2a5576d6af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2a72345b9724a36a60a5c3c53bf7e56', 'request_id': 'req-e2637c56-f8fe-48ae-8368-ba2a5576d6af'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:02.400786 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e2637c56-f8fe-48ae-8368-ba2a5576d6af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2a72345b9724a36a60a5c3c53bf7e56', 'request_id': 'req-e2637c56-f8fe-48ae-8368-ba2a5576d6af'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:02.411557 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e2637c56-f8fe-48ae-8368-ba2a5576d6af admin admin] The token cache is being invalidate because project b2a72345b9724a36a60a5c3c53bf7e56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:41:02.411847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e2637c56-f8fe-48ae-8368-ba2a5576d6af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:02.412477 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e2637c56-f8fe-48ae-8368-ba2a5576d6af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:02.414001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1305/2597] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:41:02 2022] DELETE /identity/v3/projects/b2a72345b9724a36a60a5c3c53bf7e56 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:02.418919 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81eebb-7e86-4c9c-b694-ecd021c28424 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:02.419398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81eebb-7e86-4c9c-b694-ecd021c28424 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:02.419626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e81eebb-7e86-4c9c-b694-ecd021c28424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:02.438221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9e81eebb-7e86-4c9c-b694-ecd021c28424 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:41:02.439436 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9e81eebb-7e86-4c9c-b694-ecd021c28424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:02.496312 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1293/2598] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:41:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:41:02.504619 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02463e-6602-48af-962f-960486073ffb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:02.504911 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02463e-6602-48af-962f-960486073ffb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:02.505195 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02463e-6602-48af-962f-960486073ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:02.523340 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-da02463e-6602-48af-962f-960486073ffb None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:41:02.524561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-da02463e-6602-48af-962f-960486073ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:02.532493 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1306/2599] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:41:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:41:02.536534 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81eebb-7e86-4c9c-b694-ecd021c28424 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:02.546133 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e81eebb-7e86-4c9c-b694-ecd021c28424 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:02.551219 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9cce0-a01a-4504-a92f-d6d79bf36a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:02.551865 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9cce0-a01a-4504-a92f-d6d79bf36a17 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:02.552146 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9cce0-a01a-4504-a92f-d6d79bf36a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:02.552421 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9cce0-a01a-4504-a92f-d6d79bf36a17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:02.553139 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9cce0-a01a-4504-a92f-d6d79bf36a17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:41:02.553444 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9cce0-a01a-4504-a92f-d6d79bf36a17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:02.555075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9cce0-a01a-4504-a92f-d6d79bf36a17 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:02.566915 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1294/2600] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:41:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:41:02.571241 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02463e-6602-48af-962f-960486073ffb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:02.580631 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02463e-6602-48af-962f-960486073ffb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:02.585525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511d4e5-a1ca-477a-9ef0-fc834f332c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:02.586178 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511d4e5-a1ca-477a-9ef0-fc834f332c6d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:02.586568 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511d4e5-a1ca-477a-9ef0-fc834f332c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:02.586855 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511d4e5-a1ca-477a-9ef0-fc834f332c6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:02.587547 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511d4e5-a1ca-477a-9ef0-fc834f332c6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:02.589438 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511d4e5-a1ca-477a-9ef0-fc834f332c6d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:02.615459 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1307/2601] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:41:02 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:41:02.619600 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9cce0-a01a-4504-a92f-d6d79bf36a17 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:02.629798 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9cce0-a01a-4504-a92f-d6d79bf36a17 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:02.656070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:02.687685 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b06a5-2d74-4a50-97ca-c84266dad587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:02.688394 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b06a5-2d74-4a50-97ca-c84266dad587 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:02.688671 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b06a5-2d74-4a50-97ca-c84266dad587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:02.688956 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b06a5-2d74-4a50-97ca-c84266dad587 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:02.689644 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b06a5-2d74-4a50-97ca-c84266dad587 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:02.691604 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b06a5-2d74-4a50-97ca-c84266dad587 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:02.711765 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1295/2602] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:41:02 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:41:02.715835 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511d4e5-a1ca-477a-9ef0-fc834f332c6d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:02.726748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511d4e5-a1ca-477a-9ef0-fc834f332c6d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:02.732357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9789a-6a3a-4543-b3b1-6345727091ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:02.733020 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9789a-6a3a-4543-b3b1-6345727091ab admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:02.733336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9789a-6a3a-4543-b3b1-6345727091ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:02.733616 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f9789a-6a3a-4543-b3b1-6345727091ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:02.735070 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f9789a-6a3a-4543-b3b1-6345727091ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:02.736656 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f9789a-6a3a-4543-b3b1-6345727091ab admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:02.746997 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1308/2603] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:41:02.751106 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b06a5-2d74-4a50-97ca-c84266dad587 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:02.761109 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b06a5-2d74-4a50-97ca-c84266dad587 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:02.766177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023a840f-b8fc-41aa-b2f5-86a034a5530b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:02.766876 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a840f-b8fc-41aa-b2f5-86a034a5530b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:02.767164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a840f-b8fc-41aa-b2f5-86a034a5530b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:02.767446 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023a840f-b8fc-41aa-b2f5-86a034a5530b admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:02.768163 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a840f-b8fc-41aa-b2f5-86a034a5530b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:02.769759 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023a840f-b8fc-41aa-b2f5-86a034a5530b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:02.779982 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1296/2604] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:41:02.784462 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9789a-6a3a-4543-b3b1-6345727091ab admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:02.794275 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f9789a-6a3a-4543-b3b1-6345727091ab admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:02.799488 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f66c8-4d84-4cd8-80a0-fdecd1b6dcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:02.800362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f66c8-4d84-4cd8-80a0-fdecd1b6dcea admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:02.801113 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f66c8-4d84-4cd8-80a0-fdecd1b6dcea admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:02.801392 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69f66c8-4d84-4cd8-80a0-fdecd1b6dcea admin admin] PATH_INFO: `/v3/projects/ce56cddc111549479da116bb81e8eb70/users/b2814445731c4e87bdadaa3372157f55/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:02.815551 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69f66c8-4d84-4cd8-80a0-fdecd1b6dcea admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce56cddc111549479da116bb81e8eb70, user_id=b2814445731c4e87bdadaa3372157f55, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:02.818169 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69f66c8-4d84-4cd8-80a0-fdecd1b6dcea admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:02.831302 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1309/2605] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:41:02 2022] PUT /identity/v3/projects/ce56cddc111549479da116bb81e8eb70/users/b2814445731c4e87bdadaa3372157f55/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:41:02.841464 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0ac7c-73d1-453f-993b-a1aee7233133 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:02.841758 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0ac7c-73d1-453f-993b-a1aee7233133 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:02.842042 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0ac7c-73d1-453f-993b-a1aee7233133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:02.871099 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0cf0ac7c-73d1-453f-993b-a1aee7233133 None None] MFA Rules not processed for user `b2814445731c4e87bdadaa3372157f55`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:41:02.872296 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0cf0ac7c-73d1-453f-993b-a1aee7233133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:02.935740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1297/2606] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:41:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:41:02.940171 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f66c8-4d84-4cd8-80a0-fdecd1b6dcea admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:03.001759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69f66c8-4d84-4cd8-80a0-fdecd1b6dcea admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:03.006366 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe83ca-d924-4bc2-a722-e615600c845a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:03.007042 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe83ca-d924-4bc2-a722-e615600c845a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:03.007328 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe83ca-d924-4bc2-a722-e615600c845a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:03.007603 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbe83ca-d924-4bc2-a722-e615600c845a admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:03.008290 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbe83ca-d924-4bc2-a722-e615600c845a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:03.010181 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbe83ca-d924-4bc2-a722-e615600c845a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:03.035650 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1310/2607] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:41:02 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:41:03.039843 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0ac7c-73d1-453f-993b-a1aee7233133 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:03.050946 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0ac7c-73d1-453f-993b-a1aee7233133 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:03.107532 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c552a5-7b25-4a55-8901-116cf037edd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:03.108207 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c552a5-7b25-4a55-8901-116cf037edd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:03.108494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c552a5-7b25-4a55-8901-116cf037edd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:03.109722 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c552a5-7b25-4a55-8901-116cf037edd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:03.110435 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c552a5-7b25-4a55-8901-116cf037edd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:03.112384 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c552a5-7b25-4a55-8901-116cf037edd8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:03.132118 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1298/2608] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:41:03 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:41:03.136125 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe83ca-d924-4bc2-a722-e615600c845a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:03.147132 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbe83ca-d924-4bc2-a722-e615600c845a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:03.151919 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95c538-3731-4565-8cf1-7a32a9daae1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:03.152583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95c538-3731-4565-8cf1-7a32a9daae1f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:03.152917 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95c538-3731-4565-8cf1-7a32a9daae1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:03.153187 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95c538-3731-4565-8cf1-7a32a9daae1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:03.153915 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c95c538-3731-4565-8cf1-7a32a9daae1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:03.155573 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c95c538-3731-4565-8cf1-7a32a9daae1f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:03.166501 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1311/2609] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:41:03.170200 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c552a5-7b25-4a55-8901-116cf037edd8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:03.180803 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c552a5-7b25-4a55-8901-116cf037edd8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:03.185402 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169a52a-99e1-4b59-aee9-5ff53f6169c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:03.186329 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169a52a-99e1-4b59-aee9-5ff53f6169c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:03.186600 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169a52a-99e1-4b59-aee9-5ff53f6169c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:03.186951 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169a52a-99e1-4b59-aee9-5ff53f6169c1 admin admin] PATH_INFO: `/v3/projects/018b060d16644b2db21abf58d16070ab/users/ef0cf70c754a4ef08e73f27016154438/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:03.201908 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169a52a-99e1-4b59-aee9-5ff53f6169c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=018b060d16644b2db21abf58d16070ab, user_id=ef0cf70c754a4ef08e73f27016154438, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:03.204678 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7169a52a-99e1-4b59-aee9-5ff53f6169c1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:03.217413 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1299/2610] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:41:03 2022] PUT /identity/v3/projects/018b060d16644b2db21abf58d16070ab/users/ef0cf70c754a4ef08e73f27016154438/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:03.221547 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95c538-3731-4565-8cf1-7a32a9daae1f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:03.231346 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c95c538-3731-4565-8cf1-7a32a9daae1f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:03.251821 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:03.289952 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a809d0e-483e-4fcb-a24e-0b5aa22d54be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:03.290644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a809d0e-483e-4fcb-a24e-0b5aa22d54be admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:03.290932 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a809d0e-483e-4fcb-a24e-0b5aa22d54be admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:03.291213 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a809d0e-483e-4fcb-a24e-0b5aa22d54be admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:03.291934 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a809d0e-483e-4fcb-a24e-0b5aa22d54be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:03.293522 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a809d0e-483e-4fcb-a24e-0b5aa22d54be admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:03.303638 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1312/2611] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:41:03.307978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169a52a-99e1-4b59-aee9-5ff53f6169c1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:03.314712 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:03.321164 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169a52a-99e1-4b59-aee9-5ff53f6169c1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:03.326071 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d70dba-189f-4b61-babd-2e0fbd14a461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:03.327015 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d70dba-189f-4b61-babd-2e0fbd14a461 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:03.327301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d70dba-189f-4b61-babd-2e0fbd14a461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:03.328067 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d70dba-189f-4b61-babd-2e0fbd14a461 admin admin] PATH_INFO: `/v3/projects/018b060d16644b2db21abf58d16070ab/users/ef0cf70c754a4ef08e73f27016154438/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:03.330102 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d70dba-189f-4b61-babd-2e0fbd14a461 admin admin] RBAC: Authorizing `identity:create_grant(project_id=018b060d16644b2db21abf58d16070ab, user_id=ef0cf70c754a4ef08e73f27016154438, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:03.332807 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d70dba-189f-4b61-babd-2e0fbd14a461 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:03.343516 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1300/2612] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:41:03 2022] PUT /identity/v3/projects/018b060d16644b2db21abf58d16070ab/users/ef0cf70c754a4ef08e73f27016154438/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:03.354792 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ca108-58a6-4b29-911b-6245f0ba48e0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:03.355256 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ca108-58a6-4b29-911b-6245f0ba48e0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:03.355352 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ca108-58a6-4b29-911b-6245f0ba48e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:03.382216 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-564ca108-58a6-4b29-911b-6245f0ba48e0 None None] MFA Rules not processed for user `ef0cf70c754a4ef08e73f27016154438`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:41:03.383462 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-564ca108-58a6-4b29-911b-6245f0ba48e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:03.452045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1313/2613] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:41:03 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:41:03.463364 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d70dba-189f-4b61-babd-2e0fbd14a461 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:03.465070 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c5d70dba-189f-4b61-babd-2e0fbd14a461 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:03.523190 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d70dba-189f-4b61-babd-2e0fbd14a461 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:03.526805 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc4807-b01f-4c9a-beb8-8346b1f34d76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:03.527445 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc4807-b01f-4c9a-beb8-8346b1f34d76 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:03.527734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc4807-b01f-4c9a-beb8-8346b1f34d76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:03.528020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc4807-b01f-4c9a-beb8-8346b1f34d76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:03.537629 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc4807-b01f-4c9a-beb8-8346b1f34d76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:03.539103 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dc4807-b01f-4c9a-beb8-8346b1f34d76 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:03.549210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1301/2614] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:41:03 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:41:03.614222 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ca108-58a6-4b29-911b-6245f0ba48e0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:03.675019 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564ca108-58a6-4b29-911b-6245f0ba48e0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:03.679468 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47f12b-c1ac-446e-bf1b-42edb5a0a8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:03.680331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47f12b-c1ac-446e-bf1b-42edb5a0a8fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:03.680614 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47f12b-c1ac-446e-bf1b-42edb5a0a8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:03.680886 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47f12b-c1ac-446e-bf1b-42edb5a0a8fb admin admin] PATH_INFO: `/v3/users/b2814445731c4e87bdadaa3372157f55` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:03.682057 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc47f12b-c1ac-446e-bf1b-42edb5a0a8fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2814445731c4e87bdadaa3372157f55)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:03.683981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc47f12b-c1ac-446e-bf1b-42edb5a0a8fb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:03.730163 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bc47f12b-c1ac-446e-bf1b-42edb5a0a8fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2814445731c4e87bdadaa3372157f55'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:03.731111 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bc47f12b-c1ac-446e-bf1b-42edb5a0a8fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2814445731c4e87bdadaa3372157f55'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:03.732313 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:03.744268 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bc47f12b-c1ac-446e-bf1b-42edb5a0a8fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2814445731c4e87bdadaa3372157f55'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:03.744904 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bc47f12b-c1ac-446e-bf1b-42edb5a0a8fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2814445731c4e87bdadaa3372157f55'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:03.751964 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bc47f12b-c1ac-446e-bf1b-42edb5a0a8fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2814445731c4e87bdadaa3372157f55'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:03.764289 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1314/2615] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:41:03 2022] DELETE /identity/v3/users/b2814445731c4e87bdadaa3372157f55 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:03.769729 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65403904-0fcc-4618-ac5b-a7f414c78187 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:03.770023 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65403904-0fcc-4618-ac5b-a7f414c78187 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:03.770308 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65403904-0fcc-4618-ac5b-a7f414c78187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:03.788717 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-65403904-0fcc-4618-ac5b-a7f414c78187 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:41:03.789908 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-65403904-0fcc-4618-ac5b-a7f414c78187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:03.848510 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1302/2616] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:41:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:41:03.858146 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47f12b-c1ac-446e-bf1b-42edb5a0a8fb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:03.859614 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bc47f12b-c1ac-446e-bf1b-42edb5a0a8fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:03.916413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47f12b-c1ac-446e-bf1b-42edb5a0a8fb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:03.920078 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc238d6-eeb5-4fca-a9db-073f20d5a810 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:03.920717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc238d6-eeb5-4fca-a9db-073f20d5a810 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:03.921002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc238d6-eeb5-4fca-a9db-073f20d5a810 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:03.921327 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc238d6-eeb5-4fca-a9db-073f20d5a810 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:03.930204 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc238d6-eeb5-4fca-a9db-073f20d5a810 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:03.931668 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc238d6-eeb5-4fca-a9db-073f20d5a810 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:03.941021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1315/2617] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:41:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:41:04.212833 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65403904-0fcc-4618-ac5b-a7f414c78187 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:04.214320 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-65403904-0fcc-4618-ac5b-a7f414c78187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:04.226218 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65403904-0fcc-4618-ac5b-a7f414c78187 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:04.231329 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96378a14-f84e-4fc1-bb58-fa3956d01c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:04.232189 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96378a14-f84e-4fc1-bb58-fa3956d01c68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:04.232469 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96378a14-f84e-4fc1-bb58-fa3956d01c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:04.232746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96378a14-f84e-4fc1-bb58-fa3956d01c68 admin admin] PATH_INFO: `/v3/projects/ce56cddc111549479da116bb81e8eb70` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:04.233986 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96378a14-f84e-4fc1-bb58-fa3956d01c68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce56cddc111549479da116bb81e8eb70)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:04.236123 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96378a14-f84e-4fc1-bb58-fa3956d01c68 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:04.284015 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-96378a14-f84e-4fc1-bb58-fa3956d01c68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce56cddc111549479da116bb81e8eb70', 'request_id': 'req-96378a14-f84e-4fc1-bb58-fa3956d01c68'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:04.290385 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-96378a14-f84e-4fc1-bb58-fa3956d01c68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce56cddc111549479da116bb81e8eb70', 'request_id': 'req-96378a14-f84e-4fc1-bb58-fa3956d01c68'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:04.295720 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-96378a14-f84e-4fc1-bb58-fa3956d01c68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce56cddc111549479da116bb81e8eb70', 'request_id': 'req-96378a14-f84e-4fc1-bb58-fa3956d01c68'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:04.307963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-96378a14-f84e-4fc1-bb58-fa3956d01c68 admin admin] The token cache is being invalidate because project ce56cddc111549479da116bb81e8eb70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:41:04.308260 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-96378a14-f84e-4fc1-bb58-fa3956d01c68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:04.308916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-96378a14-f84e-4fc1-bb58-fa3956d01c68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:04.310586 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1303/2618] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:41:04 2022] DELETE /identity/v3/projects/ce56cddc111549479da116bb81e8eb70 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:04.315160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc238d6-eeb5-4fca-a9db-073f20d5a810 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:04.324647 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc238d6-eeb5-4fca-a9db-073f20d5a810 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:04.381558 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b160b9-8124-4bcc-91b5-4b0178188b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:04.382444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b160b9-8124-4bcc-91b5-4b0178188b84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:04.382742 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b160b9-8124-4bcc-91b5-4b0178188b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:04.383023 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b160b9-8124-4bcc-91b5-4b0178188b84 admin admin] PATH_INFO: `/v3/users/ef0cf70c754a4ef08e73f27016154438` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:04.384207 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b160b9-8124-4bcc-91b5-4b0178188b84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef0cf70c754a4ef08e73f27016154438)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:04.386107 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b160b9-8124-4bcc-91b5-4b0178188b84 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:04.431540 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f7b160b9-8124-4bcc-91b5-4b0178188b84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef0cf70c754a4ef08e73f27016154438'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:04.432183 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f7b160b9-8124-4bcc-91b5-4b0178188b84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef0cf70c754a4ef08e73f27016154438'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:04.444475 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f7b160b9-8124-4bcc-91b5-4b0178188b84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef0cf70c754a4ef08e73f27016154438'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:04.445124 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f7b160b9-8124-4bcc-91b5-4b0178188b84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef0cf70c754a4ef08e73f27016154438'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:04.449526 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:04.453525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f7b160b9-8124-4bcc-91b5-4b0178188b84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef0cf70c754a4ef08e73f27016154438'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:04.466733 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1316/2619] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:41:04 2022] DELETE /identity/v3/users/ef0cf70c754a4ef08e73f27016154438 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:04.739721 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96378a14-f84e-4fc1-bb58-fa3956d01c68 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:04.741165 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-96378a14-f84e-4fc1-bb58-fa3956d01c68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:04.754744 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96378a14-f84e-4fc1-bb58-fa3956d01c68 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:04.767889 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:04.814711 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04461256-e6d1-41c5-b763-0b5c4c03ba6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:04.815507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04461256-e6d1-41c5-b763-0b5c4c03ba6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:04.815792 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04461256-e6d1-41c5-b763-0b5c4c03ba6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:04.816073 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04461256-e6d1-41c5-b763-0b5c4c03ba6f admin admin] PATH_INFO: `/v3/projects/018b060d16644b2db21abf58d16070ab` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:04.817239 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04461256-e6d1-41c5-b763-0b5c4c03ba6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=018b060d16644b2db21abf58d16070ab)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:04.819177 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04461256-e6d1-41c5-b763-0b5c4c03ba6f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:04.864480 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-04461256-e6d1-41c5-b763-0b5c4c03ba6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '018b060d16644b2db21abf58d16070ab', 'request_id': 'req-04461256-e6d1-41c5-b763-0b5c4c03ba6f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:04.870377 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-04461256-e6d1-41c5-b763-0b5c4c03ba6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '018b060d16644b2db21abf58d16070ab', 'request_id': 'req-04461256-e6d1-41c5-b763-0b5c4c03ba6f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:04.876022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-04461256-e6d1-41c5-b763-0b5c4c03ba6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '018b060d16644b2db21abf58d16070ab', 'request_id': 'req-04461256-e6d1-41c5-b763-0b5c4c03ba6f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:04.888773 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-04461256-e6d1-41c5-b763-0b5c4c03ba6f admin admin] The token cache is being invalidate because project 018b060d16644b2db21abf58d16070ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:41:04.889063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-04461256-e6d1-41c5-b763-0b5c4c03ba6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:04.889676 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-04461256-e6d1-41c5-b763-0b5c4c03ba6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:04.891576 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1304/2620] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:41:04 2022] DELETE /identity/v3/projects/018b060d16644b2db21abf58d16070ab => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:41:04.896640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b40831-05d8-4858-8ff9-acba3ca27706 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:04.896930 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b40831-05d8-4858-8ff9-acba3ca27706 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:04.897217 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b40831-05d8-4858-8ff9-acba3ca27706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:04.916104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e1b40831-05d8-4858-8ff9-acba3ca27706 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:41:04.918183 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e1b40831-05d8-4858-8ff9-acba3ca27706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:04.982303 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1317/2621] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:41:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:41:04.990592 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117173e3-5a41-49d3-a589-9661d1473e8e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:04.990883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117173e3-5a41-49d3-a589-9661d1473e8e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:04.991192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117173e3-5a41-49d3-a589-9661d1473e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:05.008874 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-117173e3-5a41-49d3-a589-9661d1473e8e None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:41:05.010054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-117173e3-5a41-49d3-a589-9661d1473e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:05.018823 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1305/2622] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:41:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:41:05.023295 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b40831-05d8-4858-8ff9-acba3ca27706 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:05.033583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b40831-05d8-4858-8ff9-acba3ca27706 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:05.038136 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eea741-3414-41d5-8d31-f8166fd2735f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:05.038845 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eea741-3414-41d5-8d31-f8166fd2735f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:05.039137 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eea741-3414-41d5-8d31-f8166fd2735f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:05.039419 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eea741-3414-41d5-8d31-f8166fd2735f admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:05.040151 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eea741-3414-41d5-8d31-f8166fd2735f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:41:05.040462 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eea741-3414-41d5-8d31-f8166fd2735f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:05.042056 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eea741-3414-41d5-8d31-f8166fd2735f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:05.054311 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1318/2623] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:41:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:41:05.058506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117173e3-5a41-49d3-a589-9661d1473e8e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:05.067670 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117173e3-5a41-49d3-a589-9661d1473e8e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:05.072153 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c3b19-1e1d-4218-a556-1f1679631048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:05.072783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c3b19-1e1d-4218-a556-1f1679631048 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:05.073068 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c3b19-1e1d-4218-a556-1f1679631048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:05.073347 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c3b19-1e1d-4218-a556-1f1679631048 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:05.074029 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c3b19-1e1d-4218-a556-1f1679631048 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:05.075954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c3b19-1e1d-4218-a556-1f1679631048 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:05.102872 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1306/2624] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:41:05 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:41:05.106961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eea741-3414-41d5-8d31-f8166fd2735f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:05.117415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eea741-3414-41d5-8d31-f8166fd2735f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:05.172634 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e01bb-8dbf-4047-b6e6-fff1506b7e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:05.173303 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e01bb-8dbf-4047-b6e6-fff1506b7e2f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:05.173585 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e01bb-8dbf-4047-b6e6-fff1506b7e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:05.173861 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9e01bb-8dbf-4047-b6e6-fff1506b7e2f admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:05.174589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e01bb-8dbf-4047-b6e6-fff1506b7e2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:05.176484 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9e01bb-8dbf-4047-b6e6-fff1506b7e2f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:05.195612 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1319/2625] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:41:05 2022] POST /identity/v3/users => generated 293 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:41:05.199768 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c3b19-1e1d-4218-a556-1f1679631048 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:05.211552 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c3b19-1e1d-4218-a556-1f1679631048 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:05.216302 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555071de-5fc8-4f7d-ba2c-269f53d05aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:05.216983 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555071de-5fc8-4f7d-ba2c-269f53d05aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:05.217267 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555071de-5fc8-4f7d-ba2c-269f53d05aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:05.217548 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555071de-5fc8-4f7d-ba2c-269f53d05aa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:05.218884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555071de-5fc8-4f7d-ba2c-269f53d05aa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:05.220468 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555071de-5fc8-4f7d-ba2c-269f53d05aa9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:05.230772 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1307/2626] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:41:05.234846 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e01bb-8dbf-4047-b6e6-fff1506b7e2f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:05.244709 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9e01bb-8dbf-4047-b6e6-fff1506b7e2f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:05.249365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73a6a1-3143-490d-9734-a8208a745c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:05.250058 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73a6a1-3143-490d-9734-a8208a745c90 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:05.250345 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73a6a1-3143-490d-9734-a8208a745c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:05.250661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73a6a1-3143-490d-9734-a8208a745c90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:05.251386 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d73a6a1-3143-490d-9734-a8208a745c90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:05.252976 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d73a6a1-3143-490d-9734-a8208a745c90 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:05.262742 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1320/2627] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:41:05.267293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555071de-5fc8-4f7d-ba2c-269f53d05aa9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:05.277682 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555071de-5fc8-4f7d-ba2c-269f53d05aa9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:05.282942 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb63a3d-765c-4bd4-9a8e-13406b36e7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:05.284132 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb63a3d-765c-4bd4-9a8e-13406b36e7c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:05.284498 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb63a3d-765c-4bd4-9a8e-13406b36e7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:05.284781 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb63a3d-765c-4bd4-9a8e-13406b36e7c0 admin admin] PATH_INFO: `/v3/projects/51168c485308455b8e9d8199bc8b26dc/users/7e980c4c50ed41518eaf6d541e5c0a32/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:05.300016 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb63a3d-765c-4bd4-9a8e-13406b36e7c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51168c485308455b8e9d8199bc8b26dc, user_id=7e980c4c50ed41518eaf6d541e5c0a32, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:05.302698 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb63a3d-765c-4bd4-9a8e-13406b36e7c0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:05.315548 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1308/2628] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:41:05 2022] PUT /identity/v3/projects/51168c485308455b8e9d8199bc8b26dc/users/7e980c4c50ed41518eaf6d541e5c0a32/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:05.327104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63990767-4585-4b28-b63a-39072e25b3fb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:05.327339 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63990767-4585-4b28-b63a-39072e25b3fb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:05.328146 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63990767-4585-4b28-b63a-39072e25b3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:05.355451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-63990767-4585-4b28-b63a-39072e25b3fb None None] MFA Rules not processed for user `7e980c4c50ed41518eaf6d541e5c0a32`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:41:05.356650 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-63990767-4585-4b28-b63a-39072e25b3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:05.397918 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:05.421810 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1321/2629] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:41:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:41:05.426271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb63a3d-765c-4bd4-9a8e-13406b36e7c0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:05.471988 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:05.490769 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb63a3d-765c-4bd4-9a8e-13406b36e7c0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:05.495344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfe336-6901-4cfc-ae4b-321fc44261b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:05.495989 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfe336-6901-4cfc-ae4b-321fc44261b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:05.496273 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfe336-6901-4cfc-ae4b-321fc44261b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:05.496552 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfe336-6901-4cfc-ae4b-321fc44261b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:05.497249 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dfe336-6901-4cfc-ae4b-321fc44261b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:05.499376 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dfe336-6901-4cfc-ae4b-321fc44261b6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:05.524466 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1309/2630] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:41:05 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:41:05.528511 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63990767-4585-4b28-b63a-39072e25b3fb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:05.540277 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63990767-4585-4b28-b63a-39072e25b3fb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:05.596962 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c7599-d960-4dd0-88dd-ec0b16224914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:05.597564 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c7599-d960-4dd0-88dd-ec0b16224914 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:05.597852 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c7599-d960-4dd0-88dd-ec0b16224914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:05.598131 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c7599-d960-4dd0-88dd-ec0b16224914 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:05.598872 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c7599-d960-4dd0-88dd-ec0b16224914 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:05.600778 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c7599-d960-4dd0-88dd-ec0b16224914 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:05.620230 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1322/2631] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:41:05 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:41:05.624125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfe336-6901-4cfc-ae4b-321fc44261b6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:05.634963 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfe336-6901-4cfc-ae4b-321fc44261b6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:05.639535 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96121557-c7ba-4b25-946f-44d4e89785d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:05.640980 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96121557-c7ba-4b25-946f-44d4e89785d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:05.641105 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96121557-c7ba-4b25-946f-44d4e89785d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:05.641329 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96121557-c7ba-4b25-946f-44d4e89785d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:05.642045 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96121557-c7ba-4b25-946f-44d4e89785d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:05.643680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96121557-c7ba-4b25-946f-44d4e89785d3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:05.654377 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1310/2632] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:41:05.658433 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c7599-d960-4dd0-88dd-ec0b16224914 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:05.668215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c7599-d960-4dd0-88dd-ec0b16224914 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:05.672799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27b33f-1acb-4101-881c-00873d84d0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:05.673459 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27b33f-1acb-4101-881c-00873d84d0cd admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:05.673724 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27b33f-1acb-4101-881c-00873d84d0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:05.673999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27b33f-1acb-4101-881c-00873d84d0cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:05.674756 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27b33f-1acb-4101-881c-00873d84d0cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:05.676334 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27b33f-1acb-4101-881c-00873d84d0cd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:05.685936 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1323/2633] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:41:05 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:41:05.690489 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96121557-c7ba-4b25-946f-44d4e89785d3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:05.700348 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96121557-c7ba-4b25-946f-44d4e89785d3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:05.705635 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb194e5-6bab-4b76-9c11-3861140ce156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:05.706567 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb194e5-6bab-4b76-9c11-3861140ce156 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:05.706851 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb194e5-6bab-4b76-9c11-3861140ce156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:05.707129 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb194e5-6bab-4b76-9c11-3861140ce156 admin admin] PATH_INFO: `/v3/projects/712525c5247e4d64918addd4cf1bff97/users/4c135f00354749bba2044c21025f89bd/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:05.721553 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb194e5-6bab-4b76-9c11-3861140ce156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=712525c5247e4d64918addd4cf1bff97, user_id=4c135f00354749bba2044c21025f89bd, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:05.724226 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb194e5-6bab-4b76-9c11-3861140ce156 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:05.738052 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1311/2634] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:41:05 2022] PUT /identity/v3/projects/712525c5247e4d64918addd4cf1bff97/users/4c135f00354749bba2044c21025f89bd/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:05.748287 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7b48e-a7af-41b3-b47a-411a52dbba0b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:05.748615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7b48e-a7af-41b3-b47a-411a52dbba0b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:05.748903 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7b48e-a7af-41b3-b47a-411a52dbba0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:05.776548 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e4c7b48e-a7af-41b3-b47a-411a52dbba0b None None] MFA Rules not processed for user `4c135f00354749bba2044c21025f89bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:41:05.777768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e4c7b48e-a7af-41b3-b47a-411a52dbba0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:05.842803 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1324/2635] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:41:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:41:05.847476 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb194e5-6bab-4b76-9c11-3861140ce156 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:05.909152 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb194e5-6bab-4b76-9c11-3861140ce156 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:05.913783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8b41b-8555-443a-8b24-977acefbe533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:05.914441 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8b41b-8555-443a-8b24-977acefbe533 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:05.914746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8b41b-8555-443a-8b24-977acefbe533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:05.915025 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab8b41b-8555-443a-8b24-977acefbe533 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:05.915728 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab8b41b-8555-443a-8b24-977acefbe533 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:05.917616 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab8b41b-8555-443a-8b24-977acefbe533 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:05.942311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1312/2636] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:41:05 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:41:05.946336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7b48e-a7af-41b3-b47a-411a52dbba0b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:05.957571 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7b48e-a7af-41b3-b47a-411a52dbba0b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:06.005387 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:06.015839 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2b0c6-05d1-42e1-8799-7e951250d3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:06.016509 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2b0c6-05d1-42e1-8799-7e951250d3e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:06.016795 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2b0c6-05d1-42e1-8799-7e951250d3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:06.017076 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2b0c6-05d1-42e1-8799-7e951250d3e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:06.017754 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b2b0c6-05d1-42e1-8799-7e951250d3e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:06.019682 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b2b0c6-05d1-42e1-8799-7e951250d3e2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:06.039108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1325/2637] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:41:05 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:41:06.043078 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8b41b-8555-443a-8b24-977acefbe533 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:06.054048 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab8b41b-8555-443a-8b24-977acefbe533 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:06.058707 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42ef02-6ad8-4f1a-bbf1-e3fcac200973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:06.059367 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42ef02-6ad8-4f1a-bbf1-e3fcac200973 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:06.059662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42ef02-6ad8-4f1a-bbf1-e3fcac200973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:06.059946 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42ef02-6ad8-4f1a-bbf1-e3fcac200973 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:06.060669 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce42ef02-6ad8-4f1a-bbf1-e3fcac200973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:06.063013 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce42ef02-6ad8-4f1a-bbf1-e3fcac200973 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:06.073833 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1313/2638] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:41:06.078577 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2b0c6-05d1-42e1-8799-7e951250d3e2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:06.088671 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2b0c6-05d1-42e1-8799-7e951250d3e2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:06.093511 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a8a12-c684-4dbb-9c00-889401cddc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:06.094384 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a8a12-c684-4dbb-9c00-889401cddc7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:06.094709 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a8a12-c684-4dbb-9c00-889401cddc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:06.094985 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9a8a12-c684-4dbb-9c00-889401cddc7f admin admin] PATH_INFO: `/v3/projects/e5ac05bedc484937a15c0ef7ee4661ba/users/a4ca08a81cde4f08968a81422469e8e9/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:06.110218 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a8a12-c684-4dbb-9c00-889401cddc7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5ac05bedc484937a15c0ef7ee4661ba, user_id=a4ca08a81cde4f08968a81422469e8e9, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:06.112818 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9a8a12-c684-4dbb-9c00-889401cddc7f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:06.126704 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1326/2639] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:41:06 2022] PUT /identity/v3/projects/e5ac05bedc484937a15c0ef7ee4661ba/users/a4ca08a81cde4f08968a81422469e8e9/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:06.130692 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42ef02-6ad8-4f1a-bbf1-e3fcac200973 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:06.140681 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42ef02-6ad8-4f1a-bbf1-e3fcac200973 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:06.196083 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca575b-293f-49b8-8122-7b3bbe6d6a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:06.196747 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca575b-293f-49b8-8122-7b3bbe6d6a10 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:06.197032 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca575b-293f-49b8-8122-7b3bbe6d6a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:06.197310 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca575b-293f-49b8-8122-7b3bbe6d6a10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:06.198022 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca575b-293f-49b8-8122-7b3bbe6d6a10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:06.199636 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca575b-293f-49b8-8122-7b3bbe6d6a10 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:06.209868 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1314/2640] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:41:06.214239 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a8a12-c684-4dbb-9c00-889401cddc7f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:06.224811 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9a8a12-c684-4dbb-9c00-889401cddc7f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:06.229386 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88a348-e37f-4e0e-a32f-6f5d4c689170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:06.230259 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88a348-e37f-4e0e-a32f-6f5d4c689170 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:06.230582 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88a348-e37f-4e0e-a32f-6f5d4c689170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:06.230869 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88a348-e37f-4e0e-a32f-6f5d4c689170 admin admin] PATH_INFO: `/v3/projects/e5ac05bedc484937a15c0ef7ee4661ba/users/a4ca08a81cde4f08968a81422469e8e9/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:06.232839 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce88a348-e37f-4e0e-a32f-6f5d4c689170 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5ac05bedc484937a15c0ef7ee4661ba, user_id=a4ca08a81cde4f08968a81422469e8e9, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:06.236292 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce88a348-e37f-4e0e-a32f-6f5d4c689170 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:06.246272 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1327/2641] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:41:06 2022] PUT /identity/v3/projects/e5ac05bedc484937a15c0ef7ee4661ba/users/a4ca08a81cde4f08968a81422469e8e9/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:06.256915 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4e846-f3ec-449f-85cf-bddd9009ae22 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:06.257203 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4e846-f3ec-449f-85cf-bddd9009ae22 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:06.257480 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4e846-f3ec-449f-85cf-bddd9009ae22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:06.263146 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:06.285239 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d7c4e846-f3ec-449f-85cf-bddd9009ae22 None None] MFA Rules not processed for user `a4ca08a81cde4f08968a81422469e8e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:41:06.286498 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d7c4e846-f3ec-449f-85cf-bddd9009ae22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:06.356072 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1315/2642] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:41:06 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:41:06.367354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88a348-e37f-4e0e-a32f-6f5d4c689170 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:06.369048 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ce88a348-e37f-4e0e-a32f-6f5d4c689170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:06.425898 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88a348-e37f-4e0e-a32f-6f5d4c689170 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:06.429796 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de9afa-6f60-4c46-ad5d-990c23a679da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:06.430518 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de9afa-6f60-4c46-ad5d-990c23a679da service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:06.430807 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de9afa-6f60-4c46-ad5d-990c23a679da service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:06.431085 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de9afa-6f60-4c46-ad5d-990c23a679da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:06.441260 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de9afa-6f60-4c46-ad5d-990c23a679da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:06.442724 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de9afa-6f60-4c46-ad5d-990c23a679da service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:06.451984 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1328/2643] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:41:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:41:06.946542 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4e846-f3ec-449f-85cf-bddd9009ae22 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:06.947923 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d7c4e846-f3ec-449f-85cf-bddd9009ae22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:06.960537 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4e846-f3ec-449f-85cf-bddd9009ae22 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:06.964063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514beda-4319-4cab-b0aa-48f85d4552e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:06.964701 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514beda-4319-4cab-b0aa-48f85d4552e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:06.965010 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514beda-4319-4cab-b0aa-48f85d4552e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:06.965279 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514beda-4319-4cab-b0aa-48f85d4552e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:07.021616 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f514beda-4319-4cab-b0aa-48f85d4552e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:07.023086 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f514beda-4319-4cab-b0aa-48f85d4552e7 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:07.032538 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1316/2644] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:41:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:41:10.991814 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de9afa-6f60-4c46-ad5d-990c23a679da service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:11.000653 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de9afa-6f60-4c46-ad5d-990c23a679da service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:11.004199 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e56fbd-4424-491e-a750-1808b0e89e83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:11.004828 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e56fbd-4424-491e-a750-1808b0e89e83 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:11.005110 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e56fbd-4424-491e-a750-1808b0e89e83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:11.005384 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e56fbd-4424-491e-a750-1808b0e89e83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:11.064224 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e56fbd-4424-491e-a750-1808b0e89e83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:11.065573 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e56fbd-4424-491e-a750-1808b0e89e83 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:11.074341 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1329/2645] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:41:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:41:13.127664 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514beda-4319-4cab-b0aa-48f85d4552e7 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:13.189291 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514beda-4319-4cab-b0aa-48f85d4552e7 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:13.193754 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f74cf-b43a-4679-a9be-0492272e322c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:13.194645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f74cf-b43a-4679-a9be-0492272e322c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:13.194931 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f74cf-b43a-4679-a9be-0492272e322c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:13.195210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f74cf-b43a-4679-a9be-0492272e322c admin admin] PATH_INFO: `/v3/users/7e980c4c50ed41518eaf6d541e5c0a32` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:13.196377 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89f74cf-b43a-4679-a9be-0492272e322c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e980c4c50ed41518eaf6d541e5c0a32)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:13.198292 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89f74cf-b43a-4679-a9be-0492272e322c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:13.225765 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:13.242871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a89f74cf-b43a-4679-a9be-0492272e322c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e980c4c50ed41518eaf6d541e5c0a32'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:13.249856 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a89f74cf-b43a-4679-a9be-0492272e322c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e980c4c50ed41518eaf6d541e5c0a32'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:13.263606 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a89f74cf-b43a-4679-a9be-0492272e322c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e980c4c50ed41518eaf6d541e5c0a32'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:13.264460 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a89f74cf-b43a-4679-a9be-0492272e322c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e980c4c50ed41518eaf6d541e5c0a32'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:13.265093 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a89f74cf-b43a-4679-a9be-0492272e322c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e980c4c50ed41518eaf6d541e5c0a32'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:13.278199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1317/2646] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:41:13 2022] DELETE /identity/v3/users/7e980c4c50ed41518eaf6d541e5c0a32 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:13.283060 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a68c8-9a4d-48d2-aed5-0ff44495d948 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:13.283552 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a68c8-9a4d-48d2-aed5-0ff44495d948 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:13.283688 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a68c8-9a4d-48d2-aed5-0ff44495d948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:13.300463 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d00a68c8-9a4d-48d2-aed5-0ff44495d948 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:41:13.301642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d00a68c8-9a4d-48d2-aed5-0ff44495d948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:13.342779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:13.362934 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1330/2647] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:41:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:41:13.372447 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f74cf-b43a-4679-a9be-0492272e322c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:13.373904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a89f74cf-b43a-4679-a9be-0492272e322c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:13.432145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f74cf-b43a-4679-a9be-0492272e322c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:13.435698 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cedb512-6640-4508-a5e9-bd4c2db3ceba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:13.436329 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedb512-6640-4508-a5e9-bd4c2db3ceba service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:13.436609 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedb512-6640-4508-a5e9-bd4c2db3ceba service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:13.436883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cedb512-6640-4508-a5e9-bd4c2db3ceba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:13.446242 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cedb512-6640-4508-a5e9-bd4c2db3ceba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:13.447705 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cedb512-6640-4508-a5e9-bd4c2db3ceba service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:13.456873 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1318/2648] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:41:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:41:13.722280 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a68c8-9a4d-48d2-aed5-0ff44495d948 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:13.723795 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d00a68c8-9a4d-48d2-aed5-0ff44495d948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:13.736565 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a68c8-9a4d-48d2-aed5-0ff44495d948 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:13.741357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5344e6-3bdb-490c-9a95-11ccc5b280c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:13.742258 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5344e6-3bdb-490c-9a95-11ccc5b280c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:13.742583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5344e6-3bdb-490c-9a95-11ccc5b280c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:13.742866 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5344e6-3bdb-490c-9a95-11ccc5b280c2 admin admin] PATH_INFO: `/v3/projects/51168c485308455b8e9d8199bc8b26dc` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:13.744084 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5344e6-3bdb-490c-9a95-11ccc5b280c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51168c485308455b8e9d8199bc8b26dc)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:13.745986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5344e6-3bdb-490c-9a95-11ccc5b280c2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:13.798970 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6f5344e6-3bdb-490c-9a95-11ccc5b280c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51168c485308455b8e9d8199bc8b26dc', 'request_id': 'req-6f5344e6-3bdb-490c-9a95-11ccc5b280c2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:13.806167 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6f5344e6-3bdb-490c-9a95-11ccc5b280c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51168c485308455b8e9d8199bc8b26dc', 'request_id': 'req-6f5344e6-3bdb-490c-9a95-11ccc5b280c2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:13.812648 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6f5344e6-3bdb-490c-9a95-11ccc5b280c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51168c485308455b8e9d8199bc8b26dc', 'request_id': 'req-6f5344e6-3bdb-490c-9a95-11ccc5b280c2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:13.822914 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6f5344e6-3bdb-490c-9a95-11ccc5b280c2 admin admin] The token cache is being invalidate because project 51168c485308455b8e9d8199bc8b26dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:41:13.823200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6f5344e6-3bdb-490c-9a95-11ccc5b280c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:13.823819 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6f5344e6-3bdb-490c-9a95-11ccc5b280c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:13.825343 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1331/2649] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:41:13 2022] DELETE /identity/v3/projects/51168c485308455b8e9d8199bc8b26dc => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:13.829679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cedb512-6640-4508-a5e9-bd4c2db3ceba service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:13.839312 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cedb512-6640-4508-a5e9-bd4c2db3ceba service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:13.896299 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5eec16-ec01-40bd-8565-15d12ad42b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:13.897163 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5eec16-ec01-40bd-8565-15d12ad42b60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:13.897449 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5eec16-ec01-40bd-8565-15d12ad42b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:13.897727 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5eec16-ec01-40bd-8565-15d12ad42b60 admin admin] PATH_INFO: `/v3/users/4c135f00354749bba2044c21025f89bd` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:13.899551 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5eec16-ec01-40bd-8565-15d12ad42b60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c135f00354749bba2044c21025f89bd)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:13.901457 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5eec16-ec01-40bd-8565-15d12ad42b60 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:13.947143 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5c5eec16-ec01-40bd-8565-15d12ad42b60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c135f00354749bba2044c21025f89bd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:13.949554 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:13.954820 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5c5eec16-ec01-40bd-8565-15d12ad42b60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c135f00354749bba2044c21025f89bd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:13.967226 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5c5eec16-ec01-40bd-8565-15d12ad42b60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c135f00354749bba2044c21025f89bd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:13.967853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5c5eec16-ec01-40bd-8565-15d12ad42b60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c135f00354749bba2044c21025f89bd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:13.968451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5c5eec16-ec01-40bd-8565-15d12ad42b60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c135f00354749bba2044c21025f89bd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:13.980916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1319/2650] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:41:13 2022] DELETE /identity/v3/users/4c135f00354749bba2044c21025f89bd => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:14.250058 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5344e6-3bdb-490c-9a95-11ccc5b280c2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:14.251684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6f5344e6-3bdb-490c-9a95-11ccc5b280c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:14.265145 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5344e6-3bdb-490c-9a95-11ccc5b280c2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:14.324888 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d236863-5ae8-4930-a937-c5e8e5f8dacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:14.325716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d236863-5ae8-4930-a937-c5e8e5f8dacb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:14.326001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d236863-5ae8-4930-a937-c5e8e5f8dacb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:14.326277 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d236863-5ae8-4930-a937-c5e8e5f8dacb admin admin] PATH_INFO: `/v3/projects/712525c5247e4d64918addd4cf1bff97` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:14.327498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d236863-5ae8-4930-a937-c5e8e5f8dacb admin admin] RBAC: Authorizing `identity:delete_project(project_id=712525c5247e4d64918addd4cf1bff97)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:14.330149 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d236863-5ae8-4930-a937-c5e8e5f8dacb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:14.376712 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0d236863-5ae8-4930-a937-c5e8e5f8dacb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '712525c5247e4d64918addd4cf1bff97', 'request_id': 'req-0d236863-5ae8-4930-a937-c5e8e5f8dacb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:14.383908 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0d236863-5ae8-4930-a937-c5e8e5f8dacb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '712525c5247e4d64918addd4cf1bff97', 'request_id': 'req-0d236863-5ae8-4930-a937-c5e8e5f8dacb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:14.389968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0d236863-5ae8-4930-a937-c5e8e5f8dacb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '712525c5247e4d64918addd4cf1bff97', 'request_id': 'req-0d236863-5ae8-4930-a937-c5e8e5f8dacb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:14.397125 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:14.402026 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0d236863-5ae8-4930-a937-c5e8e5f8dacb admin admin] The token cache is being invalidate because project 712525c5247e4d64918addd4cf1bff97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:41:14.402599 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0d236863-5ae8-4930-a937-c5e8e5f8dacb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:14.403602 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0d236863-5ae8-4930-a937-c5e8e5f8dacb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:14.405270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1332/2651] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:41:14 2022] DELETE /identity/v3/projects/712525c5247e4d64918addd4cf1bff97 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:14.409726 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5eec16-ec01-40bd-8565-15d12ad42b60 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:14.411152 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5c5eec16-ec01-40bd-8565-15d12ad42b60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:14.423535 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5eec16-ec01-40bd-8565-15d12ad42b60 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:14.478958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f4c1ed-66f7-464a-aa8b-b87f9803f66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:14.479820 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4c1ed-66f7-464a-aa8b-b87f9803f66d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:14.480103 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4c1ed-66f7-464a-aa8b-b87f9803f66d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:14.480455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4c1ed-66f7-464a-aa8b-b87f9803f66d admin admin] PATH_INFO: `/v3/users/a4ca08a81cde4f08968a81422469e8e9` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:14.481626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f4c1ed-66f7-464a-aa8b-b87f9803f66d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4ca08a81cde4f08968a81422469e8e9)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:14.483564 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f4c1ed-66f7-464a-aa8b-b87f9803f66d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:14.528985 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e3f4c1ed-66f7-464a-aa8b-b87f9803f66d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4ca08a81cde4f08968a81422469e8e9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:14.535924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e3f4c1ed-66f7-464a-aa8b-b87f9803f66d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4ca08a81cde4f08968a81422469e8e9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:14.548124 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e3f4c1ed-66f7-464a-aa8b-b87f9803f66d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4ca08a81cde4f08968a81422469e8e9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:14.548794 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e3f4c1ed-66f7-464a-aa8b-b87f9803f66d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4ca08a81cde4f08968a81422469e8e9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:14.549403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e3f4c1ed-66f7-464a-aa8b-b87f9803f66d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4ca08a81cde4f08968a81422469e8e9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:14.561468 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1320/2652] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:41:14 2022] DELETE /identity/v3/users/a4ca08a81cde4f08968a81422469e8e9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:14.843895 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d236863-5ae8-4930-a937-c5e8e5f8dacb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:14.846000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0d236863-5ae8-4930-a937-c5e8e5f8dacb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:14.858870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d236863-5ae8-4930-a937-c5e8e5f8dacb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:14.916521 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e679c-d5db-462e-8459-3f352782ae41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:14.917380 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e679c-d5db-462e-8459-3f352782ae41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:14.917666 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e679c-d5db-462e-8459-3f352782ae41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:14.917943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e679c-d5db-462e-8459-3f352782ae41 admin admin] PATH_INFO: `/v3/projects/e5ac05bedc484937a15c0ef7ee4661ba` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:14.919143 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e679c-d5db-462e-8459-3f352782ae41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5ac05bedc484937a15c0ef7ee4661ba)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:14.921044 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e679c-d5db-462e-8459-3f352782ae41 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:14.966234 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3e4e679c-d5db-462e-8459-3f352782ae41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5ac05bedc484937a15c0ef7ee4661ba', 'request_id': 'req-3e4e679c-d5db-462e-8459-3f352782ae41'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:14.973381 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3e4e679c-d5db-462e-8459-3f352782ae41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5ac05bedc484937a15c0ef7ee4661ba', 'request_id': 'req-3e4e679c-d5db-462e-8459-3f352782ae41'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:14.979461 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3e4e679c-d5db-462e-8459-3f352782ae41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5ac05bedc484937a15c0ef7ee4661ba', 'request_id': 'req-3e4e679c-d5db-462e-8459-3f352782ae41'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:14.990216 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3e4e679c-d5db-462e-8459-3f352782ae41 admin admin] The token cache is being invalidate because project e5ac05bedc484937a15c0ef7ee4661ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:41:14.990535 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3e4e679c-d5db-462e-8459-3f352782ae41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:14.991175 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3e4e679c-d5db-462e-8459-3f352782ae41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:14.992692 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1333/2653] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:41:14 2022] DELETE /identity/v3/projects/e5ac05bedc484937a15c0ef7ee4661ba => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:14.998645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ea5e0-6b67-42dd-8e7c-c33b7849a74f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:14.998943 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ea5e0-6b67-42dd-8e7c-c33b7849a74f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:14.999229 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ea5e0-6b67-42dd-8e7c-c33b7849a74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:15.017334 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-656ea5e0-6b67-42dd-8e7c-c33b7849a74f None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:41:15.018566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-656ea5e0-6b67-42dd-8e7c-c33b7849a74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:15.060461 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:15.079482 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1321/2654] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:41:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:41:15.088883 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a3f31-a1ce-4d9e-8132-cb0da8c44867 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:15.089120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a3f31-a1ce-4d9e-8132-cb0da8c44867 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:15.089405 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a3f31-a1ce-4d9e-8132-cb0da8c44867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:15.107561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d53a3f31-a1ce-4d9e-8132-cb0da8c44867 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:41:15.108765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d53a3f31-a1ce-4d9e-8132-cb0da8c44867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:15.117144 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1334/2655] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:41:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:41:15.121258 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ea5e0-6b67-42dd-8e7c-c33b7849a74f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:15.131656 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ea5e0-6b67-42dd-8e7c-c33b7849a74f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:15.136289 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bdec22-2c29-4cf7-b632-30f4db30597e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:15.136940 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdec22-2c29-4cf7-b632-30f4db30597e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:15.137229 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdec22-2c29-4cf7-b632-30f4db30597e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:15.137511 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bdec22-2c29-4cf7-b632-30f4db30597e admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:15.138238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bdec22-2c29-4cf7-b632-30f4db30597e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:41:15.138592 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bdec22-2c29-4cf7-b632-30f4db30597e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:15.140185 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bdec22-2c29-4cf7-b632-30f4db30597e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:15.152090 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1322/2656] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:41:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:41:15.156497 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a3f31-a1ce-4d9e-8132-cb0da8c44867 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:15.165691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a3f31-a1ce-4d9e-8132-cb0da8c44867 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:15.170243 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77368fd-2206-499b-8ad2-7a890b46021c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:15.170924 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77368fd-2206-499b-8ad2-7a890b46021c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:15.171210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77368fd-2206-499b-8ad2-7a890b46021c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:15.171490 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77368fd-2206-499b-8ad2-7a890b46021c admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:15.172205 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77368fd-2206-499b-8ad2-7a890b46021c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:15.175000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77368fd-2206-499b-8ad2-7a890b46021c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:15.200860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1335/2657] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:41:15 2022] POST /identity/v3/projects => generated 393 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:41:15.204809 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bdec22-2c29-4cf7-b632-30f4db30597e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:15.215290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bdec22-2c29-4cf7-b632-30f4db30597e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:15.271663 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c1f38-2115-402a-a0d3-6c8d57562dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:15.272324 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c1f38-2115-402a-a0d3-6c8d57562dd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:15.272615 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c1f38-2115-402a-a0d3-6c8d57562dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:15.272893 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0c1f38-2115-402a-a0d3-6c8d57562dd9 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:15.273598 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c1f38-2115-402a-a0d3-6c8d57562dd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:15.275543 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0c1f38-2115-402a-a0d3-6c8d57562dd9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:15.296182 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1323/2658] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:41:15 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:41:15.301537 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77368fd-2206-499b-8ad2-7a890b46021c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:15.311936 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77368fd-2206-499b-8ad2-7a890b46021c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:15.316661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1103661-d498-4be3-9945-0d2247f748b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:15.317324 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1103661-d498-4be3-9945-0d2247f748b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:15.317608 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1103661-d498-4be3-9945-0d2247f748b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:15.317885 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1103661-d498-4be3-9945-0d2247f748b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:15.318641 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1103661-d498-4be3-9945-0d2247f748b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:15.320310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1103661-d498-4be3-9945-0d2247f748b2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:15.332035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1336/2659] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:41:15.339433 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c1f38-2115-402a-a0d3-6c8d57562dd9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:15.353040 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0c1f38-2115-402a-a0d3-6c8d57562dd9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:15.357720 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a3251a-4191-4d25-8fce-3f8a869c8d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:15.358413 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3251a-4191-4d25-8fce-3f8a869c8d3d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:15.358728 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3251a-4191-4d25-8fce-3f8a869c8d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:15.359007 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3251a-4191-4d25-8fce-3f8a869c8d3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:15.359728 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a3251a-4191-4d25-8fce-3f8a869c8d3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:15.361911 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a3251a-4191-4d25-8fce-3f8a869c8d3d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:15.372127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1324/2660] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:41:15.376459 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1103661-d498-4be3-9945-0d2247f748b2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:15.386535 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1103661-d498-4be3-9945-0d2247f748b2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:15.391335 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb250d-6590-4c73-8e98-63aabc570a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:15.392249 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb250d-6590-4c73-8e98-63aabc570a62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:15.392524 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb250d-6590-4c73-8e98-63aabc570a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:15.392797 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb250d-6590-4c73-8e98-63aabc570a62 admin admin] PATH_INFO: `/v3/projects/6fb0993542c941ac8507001bbfc0d897/users/db3c53bfcbdc4309b17378d37c4a9ddd/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:15.407760 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb250d-6590-4c73-8e98-63aabc570a62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fb0993542c941ac8507001bbfc0d897, user_id=db3c53bfcbdc4309b17378d37c4a9ddd, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:15.410357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb250d-6590-4c73-8e98-63aabc570a62 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:15.424444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1337/2661] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:41:15 2022] PUT /identity/v3/projects/6fb0993542c941ac8507001bbfc0d897/users/db3c53bfcbdc4309b17378d37c4a9ddd/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:15.435559 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42de57-643c-4fee-82c2-203f6f29699f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:15.436036 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42de57-643c-4fee-82c2-203f6f29699f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:15.436131 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42de57-643c-4fee-82c2-203f6f29699f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:15.462547 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6e42de57-643c-4fee-82c2-203f6f29699f None None] MFA Rules not processed for user `db3c53bfcbdc4309b17378d37c4a9ddd`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:41:15.463754 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6e42de57-643c-4fee-82c2-203f6f29699f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:15.526652 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1325/2662] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:41:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:41:15.531139 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb250d-6590-4c73-8e98-63aabc570a62 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:15.558833 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:15.595938 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb250d-6590-4c73-8e98-63aabc570a62 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:15.600628 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a552cf-805c-4db0-9b4e-877ec9459187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:15.601281 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a552cf-805c-4db0-9b4e-877ec9459187 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:15.601570 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a552cf-805c-4db0-9b4e-877ec9459187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:15.601850 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a552cf-805c-4db0-9b4e-877ec9459187 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:15.602567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a552cf-805c-4db0-9b4e-877ec9459187 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:15.604455 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a552cf-805c-4db0-9b4e-877ec9459187 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:15.629878 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1338/2663] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:41:15 2022] POST /identity/v3/projects => generated 393 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:41:15.634189 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42de57-643c-4fee-82c2-203f6f29699f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:15.647522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42de57-643c-4fee-82c2-203f6f29699f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:15.692561 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:15.711146 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba9d90-7c29-4b5c-a39b-30ef2e13b969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:15.711824 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba9d90-7c29-4b5c-a39b-30ef2e13b969 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:15.712108 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba9d90-7c29-4b5c-a39b-30ef2e13b969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:15.712384 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba9d90-7c29-4b5c-a39b-30ef2e13b969 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:15.713085 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba9d90-7c29-4b5c-a39b-30ef2e13b969 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:15.715066 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba9d90-7c29-4b5c-a39b-30ef2e13b969 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:15.739253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1326/2664] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:41:15 2022] POST /identity/v3/users => generated 303 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:41:15.743479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a552cf-805c-4db0-9b4e-877ec9459187 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:15.754277 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a552cf-805c-4db0-9b4e-877ec9459187 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:15.758887 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f03e25-9202-4ed9-92d5-584eec7f0580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:15.759543 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f03e25-9202-4ed9-92d5-584eec7f0580 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:15.759829 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f03e25-9202-4ed9-92d5-584eec7f0580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:15.760105 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f03e25-9202-4ed9-92d5-584eec7f0580 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:15.760834 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f03e25-9202-4ed9-92d5-584eec7f0580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:15.762468 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f03e25-9202-4ed9-92d5-584eec7f0580 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:15.773972 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1339/2665] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:41:15.778035 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba9d90-7c29-4b5c-a39b-30ef2e13b969 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:15.788501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba9d90-7c29-4b5c-a39b-30ef2e13b969 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:15.793108 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1f44c-d2c6-44ee-9f1f-3408550f77e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:15.793971 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1f44c-d2c6-44ee-9f1f-3408550f77e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:15.794259 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1f44c-d2c6-44ee-9f1f-3408550f77e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:15.794596 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1f44c-d2c6-44ee-9f1f-3408550f77e2 admin admin] PATH_INFO: `/v3/projects/adf00d8bf1f64965b9eb1eac276548f8/users/9733de3edc8249adb662f538eac0e62e/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:15.810521 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d1f44c-d2c6-44ee-9f1f-3408550f77e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adf00d8bf1f64965b9eb1eac276548f8, user_id=9733de3edc8249adb662f538eac0e62e, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:15.813192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d1f44c-d2c6-44ee-9f1f-3408550f77e2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:15.826780 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1327/2666] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:41:15 2022] PUT /identity/v3/projects/adf00d8bf1f64965b9eb1eac276548f8/users/9733de3edc8249adb662f538eac0e62e/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:15.831572 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f03e25-9202-4ed9-92d5-584eec7f0580 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:15.841419 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f03e25-9202-4ed9-92d5-584eec7f0580 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:15.897546 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67929cb3-61e0-4b5e-ab21-4ea22707122b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:15.898128 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67929cb3-61e0-4b5e-ab21-4ea22707122b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:15.898412 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67929cb3-61e0-4b5e-ab21-4ea22707122b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:15.898716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67929cb3-61e0-4b5e-ab21-4ea22707122b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:15.899433 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67929cb3-61e0-4b5e-ab21-4ea22707122b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:15.901034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67929cb3-61e0-4b5e-ab21-4ea22707122b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:15.911399 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1340/2667] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:41:15.915574 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1f44c-d2c6-44ee-9f1f-3408550f77e2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:15.926239 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1f44c-d2c6-44ee-9f1f-3408550f77e2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:15.930804 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af427b13-0362-4ba2-a4c6-efd0665969ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:15.931679 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af427b13-0362-4ba2-a4c6-efd0665969ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:15.931964 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af427b13-0362-4ba2-a4c6-efd0665969ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:15.932241 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af427b13-0362-4ba2-a4c6-efd0665969ce admin admin] PATH_INFO: `/v3/projects/adf00d8bf1f64965b9eb1eac276548f8/users/9733de3edc8249adb662f538eac0e62e/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:15.934179 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af427b13-0362-4ba2-a4c6-efd0665969ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=adf00d8bf1f64965b9eb1eac276548f8, user_id=9733de3edc8249adb662f538eac0e62e, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:15.936898 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af427b13-0362-4ba2-a4c6-efd0665969ce admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:15.947516 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1328/2668] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:41:15 2022] PUT /identity/v3/projects/adf00d8bf1f64965b9eb1eac276548f8/users/9733de3edc8249adb662f538eac0e62e/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:41:15.959532 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38d37c-bef3-4b1c-b358-3a5eb1e63ce3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:15.959821 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38d37c-bef3-4b1c-b358-3a5eb1e63ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:15.960098 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38d37c-bef3-4b1c-b358-3a5eb1e63ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:15.987087 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7c38d37c-bef3-4b1c-b358-3a5eb1e63ce3 None None] MFA Rules not processed for user `9733de3edc8249adb662f538eac0e62e`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:41:15.988291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7c38d37c-bef3-4b1c-b358-3a5eb1e63ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:16.059049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1341/2669] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:41:15 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:41:16.070416 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af427b13-0362-4ba2-a4c6-efd0665969ce admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:16.072125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-af427b13-0362-4ba2-a4c6-efd0665969ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:16.129099 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af427b13-0362-4ba2-a4c6-efd0665969ce admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:16.132989 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3778ff-1d85-41f4-92e3-50c58bd20970 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:16.133632 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3778ff-1d85-41f4-92e3-50c58bd20970 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:16.133917 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3778ff-1d85-41f4-92e3-50c58bd20970 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:16.134196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3778ff-1d85-41f4-92e3-50c58bd20970 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:16.143919 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3778ff-1d85-41f4-92e3-50c58bd20970 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:16.145341 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3778ff-1d85-41f4-92e3-50c58bd20970 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:16.154917 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1329/2670] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:41:16 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:41:16.200362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38d37c-bef3-4b1c-b358-3a5eb1e63ce3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:16.201737 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7c38d37c-bef3-4b1c-b358-3a5eb1e63ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:16.214709 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c38d37c-bef3-4b1c-b358-3a5eb1e63ce3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:16.218245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b713f-4911-4a1e-93ce-89d4ac7d6214 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:16.218988 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b713f-4911-4a1e-93ce-89d4ac7d6214 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:16.219273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b713f-4911-4a1e-93ce-89d4ac7d6214 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:16.219596 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b713f-4911-4a1e-93ce-89d4ac7d6214 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:16.236051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:16.276366 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b713f-4911-4a1e-93ce-89d4ac7d6214 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:16.277791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b713f-4911-4a1e-93ce-89d4ac7d6214 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:16.286677 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1342/2671] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:41:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:41:19.560510 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3778ff-1d85-41f4-92e3-50c58bd20970 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:19.622468 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3778ff-1d85-41f4-92e3-50c58bd20970 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:19.627747 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f419866-04af-433d-a8e3-8cbd2e1f1fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:19.628608 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f419866-04af-433d-a8e3-8cbd2e1f1fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:19.628887 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f419866-04af-433d-a8e3-8cbd2e1f1fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:19.629159 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f419866-04af-433d-a8e3-8cbd2e1f1fb0 admin admin] PATH_INFO: `/v3/users/db3c53bfcbdc4309b17378d37c4a9ddd` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:19.630338 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f419866-04af-433d-a8e3-8cbd2e1f1fb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db3c53bfcbdc4309b17378d37c4a9ddd)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:19.632323 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f419866-04af-433d-a8e3-8cbd2e1f1fb0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:19.671451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:19.679461 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6f419866-04af-433d-a8e3-8cbd2e1f1fb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db3c53bfcbdc4309b17378d37c4a9ddd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:19.686973 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6f419866-04af-433d-a8e3-8cbd2e1f1fb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db3c53bfcbdc4309b17378d37c4a9ddd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:19.699865 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6f419866-04af-433d-a8e3-8cbd2e1f1fb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db3c53bfcbdc4309b17378d37c4a9ddd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:19.700530 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6f419866-04af-433d-a8e3-8cbd2e1f1fb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db3c53bfcbdc4309b17378d37c4a9ddd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:19.701136 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6f419866-04af-433d-a8e3-8cbd2e1f1fb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db3c53bfcbdc4309b17378d37c4a9ddd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:19.713167 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1330/2672] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:41:19 2022] DELETE /identity/v3/users/db3c53bfcbdc4309b17378d37c4a9ddd => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:19.717677 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc351888-5bd2-4533-a9c1-cb29292b9f52 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:19.718147 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc351888-5bd2-4533-a9c1-cb29292b9f52 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:19.718257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc351888-5bd2-4533-a9c1-cb29292b9f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:19.735957 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dc351888-5bd2-4533-a9c1-cb29292b9f52 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:41:19.737301 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dc351888-5bd2-4533-a9c1-cb29292b9f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:19.796690 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1343/2673] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:41:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:41:19.807084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f419866-04af-433d-a8e3-8cbd2e1f1fb0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:19.808474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6f419866-04af-433d-a8e3-8cbd2e1f1fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:19.866402 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f419866-04af-433d-a8e3-8cbd2e1f1fb0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:19.870033 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1dc02-e498-495d-9589-b668a3c83bb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:19.870709 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1dc02-e498-495d-9589-b668a3c83bb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:19.870994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1dc02-e498-495d-9589-b668a3c83bb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:19.871274 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1dc02-e498-495d-9589-b668a3c83bb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:19.880698 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1dc02-e498-495d-9589-b668a3c83bb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:19.882065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1dc02-e498-495d-9589-b668a3c83bb8 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:19.891897 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1331/2674] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:41:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:41:20.173377 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc351888-5bd2-4533-a9c1-cb29292b9f52 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:20.175137 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dc351888-5bd2-4533-a9c1-cb29292b9f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:20.191317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc351888-5bd2-4533-a9c1-cb29292b9f52 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:20.196906 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a3b37-4495-4f52-9310-ac6231356062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:20.197666 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a3b37-4495-4f52-9310-ac6231356062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:20.197952 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a3b37-4495-4f52-9310-ac6231356062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:20.198226 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5a3b37-4495-4f52-9310-ac6231356062 admin admin] PATH_INFO: `/v3/projects/6fb0993542c941ac8507001bbfc0d897` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:20.199440 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5a3b37-4495-4f52-9310-ac6231356062 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fb0993542c941ac8507001bbfc0d897)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:20.201322 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5a3b37-4495-4f52-9310-ac6231356062 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:20.248430 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2b5a3b37-4495-4f52-9310-ac6231356062 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fb0993542c941ac8507001bbfc0d897', 'request_id': 'req-2b5a3b37-4495-4f52-9310-ac6231356062'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:20.255788 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2b5a3b37-4495-4f52-9310-ac6231356062 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fb0993542c941ac8507001bbfc0d897', 'request_id': 'req-2b5a3b37-4495-4f52-9310-ac6231356062'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:20.261886 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2b5a3b37-4495-4f52-9310-ac6231356062 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fb0993542c941ac8507001bbfc0d897', 'request_id': 'req-2b5a3b37-4495-4f52-9310-ac6231356062'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:20.273831 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2b5a3b37-4495-4f52-9310-ac6231356062 admin admin] The token cache is being invalidate because project 6fb0993542c941ac8507001bbfc0d897 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:41:20.274117 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2b5a3b37-4495-4f52-9310-ac6231356062 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:20.274760 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-2b5a3b37-4495-4f52-9310-ac6231356062 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:20.276277 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1344/2675] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:41:20 2022] DELETE /identity/v3/projects/6fb0993542c941ac8507001bbfc0d897 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:20.280214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1dc02-e498-495d-9589-b668a3c83bb8 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:20.290494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1dc02-e498-495d-9589-b668a3c83bb8 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:20.347524 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d58fb-fca7-462d-bf61-86f34ea6d1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:20.348892 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d58fb-fca7-462d-bf61-86f34ea6d1a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:20.349198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d58fb-fca7-462d-bf61-86f34ea6d1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:20.349481 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d58fb-fca7-462d-bf61-86f34ea6d1a4 admin admin] PATH_INFO: `/v3/users/9733de3edc8249adb662f538eac0e62e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:20.350725 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d58fb-fca7-462d-bf61-86f34ea6d1a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9733de3edc8249adb662f538eac0e62e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:20.352648 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d58fb-fca7-462d-bf61-86f34ea6d1a4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:20.399041 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9f6d58fb-fca7-462d-bf61-86f34ea6d1a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9733de3edc8249adb662f538eac0e62e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:20.406049 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9f6d58fb-fca7-462d-bf61-86f34ea6d1a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9733de3edc8249adb662f538eac0e62e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:20.412116 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:20.419904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9f6d58fb-fca7-462d-bf61-86f34ea6d1a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9733de3edc8249adb662f538eac0e62e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:20.420575 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9f6d58fb-fca7-462d-bf61-86f34ea6d1a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9733de3edc8249adb662f538eac0e62e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:20.421526 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9f6d58fb-fca7-462d-bf61-86f34ea6d1a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9733de3edc8249adb662f538eac0e62e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:20.433656 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1332/2676] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:41:20 2022] DELETE /identity/v3/users/9733de3edc8249adb662f538eac0e62e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:41:20.699793 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a3b37-4495-4f52-9310-ac6231356062 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:20.701207 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2b5a3b37-4495-4f52-9310-ac6231356062 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:20.714731 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5a3b37-4495-4f52-9310-ac6231356062 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:20.756786 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:20.775368 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e98634-be24-4004-8dd2-fedcd14feff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:20.776239 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e98634-be24-4004-8dd2-fedcd14feff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:20.776517 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e98634-be24-4004-8dd2-fedcd14feff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:20.776791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e98634-be24-4004-8dd2-fedcd14feff7 admin admin] PATH_INFO: `/v3/projects/adf00d8bf1f64965b9eb1eac276548f8` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:20.777947 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e98634-be24-4004-8dd2-fedcd14feff7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adf00d8bf1f64965b9eb1eac276548f8)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:20.779864 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e98634-be24-4004-8dd2-fedcd14feff7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:20.826702 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-14e98634-be24-4004-8dd2-fedcd14feff7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adf00d8bf1f64965b9eb1eac276548f8', 'request_id': 'req-14e98634-be24-4004-8dd2-fedcd14feff7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:20.833896 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-14e98634-be24-4004-8dd2-fedcd14feff7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adf00d8bf1f64965b9eb1eac276548f8', 'request_id': 'req-14e98634-be24-4004-8dd2-fedcd14feff7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:20.840200 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-14e98634-be24-4004-8dd2-fedcd14feff7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adf00d8bf1f64965b9eb1eac276548f8', 'request_id': 'req-14e98634-be24-4004-8dd2-fedcd14feff7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:20.851159 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-14e98634-be24-4004-8dd2-fedcd14feff7 admin admin] The token cache is being invalidate because project adf00d8bf1f64965b9eb1eac276548f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:41:20.851451 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-14e98634-be24-4004-8dd2-fedcd14feff7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:20.852085 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-14e98634-be24-4004-8dd2-fedcd14feff7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:20.853625 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1345/2677] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:41:20 2022] DELETE /identity/v3/projects/adf00d8bf1f64965b9eb1eac276548f8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:20.858506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfeef7c-2a41-4bd4-af66-2822368d0a61 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:20.858992 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfeef7c-2a41-4bd4-af66-2822368d0a61 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:20.859116 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfeef7c-2a41-4bd4-af66-2822368d0a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:20.877270 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-dcfeef7c-2a41-4bd4-af66-2822368d0a61 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:41:20.878486 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dcfeef7c-2a41-4bd4-af66-2822368d0a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:20.937057 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1333/2678] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:41:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:41:20.945017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229f3df-b267-4a9d-8f54-34cccd2e860d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:20.945282 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229f3df-b267-4a9d-8f54-34cccd2e860d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:20.945569 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229f3df-b267-4a9d-8f54-34cccd2e860d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:20.965127 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b229f3df-b267-4a9d-8f54-34cccd2e860d None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:41:20.966337 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b229f3df-b267-4a9d-8f54-34cccd2e860d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:20.975076 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1346/2679] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:41:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:41:20.979334 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfeef7c-2a41-4bd4-af66-2822368d0a61 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:20.989455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfeef7c-2a41-4bd4-af66-2822368d0a61 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:20.994153 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338f098-c39f-447b-bbef-085e65ee317a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:20.994862 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338f098-c39f-447b-bbef-085e65ee317a admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:20.995148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338f098-c39f-447b-bbef-085e65ee317a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:20.995430 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338f098-c39f-447b-bbef-085e65ee317a admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:20.996167 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9338f098-c39f-447b-bbef-085e65ee317a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:41:20.996484 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9338f098-c39f-447b-bbef-085e65ee317a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:20.998100 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9338f098-c39f-447b-bbef-085e65ee317a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:21.014461 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1334/2680] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:41:21.015549 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229f3df-b267-4a9d-8f54-34cccd2e860d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:21.025553 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229f3df-b267-4a9d-8f54-34cccd2e860d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:21.030213 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77dddb8-e75c-4ee2-822f-e05b69b3b6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:21.030906 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77dddb8-e75c-4ee2-822f-e05b69b3b6cc admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:21.031194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77dddb8-e75c-4ee2-822f-e05b69b3b6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:21.031471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77dddb8-e75c-4ee2-822f-e05b69b3b6cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:21.032842 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77dddb8-e75c-4ee2-822f-e05b69b3b6cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:21.034912 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77dddb8-e75c-4ee2-822f-e05b69b3b6cc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:21.063953 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1347/2681] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:41:21 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:41:21.065133 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338f098-c39f-447b-bbef-085e65ee317a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:21.075215 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338f098-c39f-447b-bbef-085e65ee317a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:21.131411 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997bb9a4-d7ba-4c0c-8d59-0476a0c728eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:21.132083 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997bb9a4-d7ba-4c0c-8d59-0476a0c728eb admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:21.132365 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997bb9a4-d7ba-4c0c-8d59-0476a0c728eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:21.132645 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997bb9a4-d7ba-4c0c-8d59-0476a0c728eb admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:21.133325 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997bb9a4-d7ba-4c0c-8d59-0476a0c728eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:21.135271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997bb9a4-d7ba-4c0c-8d59-0476a0c728eb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:21.156292 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1335/2682] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:41:21 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:41:21.160714 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77dddb8-e75c-4ee2-822f-e05b69b3b6cc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:21.171634 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77dddb8-e75c-4ee2-822f-e05b69b3b6cc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:21.177030 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6af60f-31e0-4273-9c03-6c4e5fe0a29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:21.177717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6af60f-31e0-4273-9c03-6c4e5fe0a29d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:21.178000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6af60f-31e0-4273-9c03-6c4e5fe0a29d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:21.178272 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6af60f-31e0-4273-9c03-6c4e5fe0a29d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:21.179021 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6af60f-31e0-4273-9c03-6c4e5fe0a29d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:21.180653 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6af60f-31e0-4273-9c03-6c4e5fe0a29d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:21.191740 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1348/2683] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:41:21.195640 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997bb9a4-d7ba-4c0c-8d59-0476a0c728eb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:21.205752 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997bb9a4-d7ba-4c0c-8d59-0476a0c728eb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:21.211520 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767218b4-2a8a-4038-b907-b1b622e48a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:21.212544 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767218b4-2a8a-4038-b907-b1b622e48a6d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:21.212835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767218b4-2a8a-4038-b907-b1b622e48a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:21.213119 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767218b4-2a8a-4038-b907-b1b622e48a6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:21.213845 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767218b4-2a8a-4038-b907-b1b622e48a6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:21.215473 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767218b4-2a8a-4038-b907-b1b622e48a6d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:21.225987 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1336/2684] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:41:21.230029 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6af60f-31e0-4273-9c03-6c4e5fe0a29d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:21.240626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6af60f-31e0-4273-9c03-6c4e5fe0a29d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:21.245372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66dde58-c3bf-4797-8aad-8a4e85c65f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:21.246293 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66dde58-c3bf-4797-8aad-8a4e85c65f8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:21.246601 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66dde58-c3bf-4797-8aad-8a4e85c65f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:21.246878 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66dde58-c3bf-4797-8aad-8a4e85c65f8b admin admin] PATH_INFO: `/v3/projects/6fa35cc45e864e6b898e72ad724cad82/users/0fb1508d9f874110a672d5a1f27d64d1/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:21.262897 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66dde58-c3bf-4797-8aad-8a4e85c65f8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fa35cc45e864e6b898e72ad724cad82, user_id=0fb1508d9f874110a672d5a1f27d64d1, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:21.265515 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66dde58-c3bf-4797-8aad-8a4e85c65f8b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:21.278925 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1349/2685] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:41:21 2022] PUT /identity/v3/projects/6fa35cc45e864e6b898e72ad724cad82/users/0fb1508d9f874110a672d5a1f27d64d1/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:21.290344 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86eb32-f5eb-4978-a193-f4c6bed6e79c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:21.290675 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86eb32-f5eb-4978-a193-f4c6bed6e79c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:21.290976 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86eb32-f5eb-4978-a193-f4c6bed6e79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:21.318113 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6e86eb32-f5eb-4978-a193-f4c6bed6e79c None None] MFA Rules not processed for user `0fb1508d9f874110a672d5a1f27d64d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:41:21.319374 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6e86eb32-f5eb-4978-a193-f4c6bed6e79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:21.351517 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:21.385845 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1337/2686] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:41:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:41:21.390325 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66dde58-c3bf-4797-8aad-8a4e85c65f8b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:21.452255 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66dde58-c3bf-4797-8aad-8a4e85c65f8b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:21.456847 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1f6da-57b1-4a67-a7e7-d541cbd820d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:21.457500 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1f6da-57b1-4a67-a7e7-d541cbd820d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:21.457789 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1f6da-57b1-4a67-a7e7-d541cbd820d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:21.458070 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1f6da-57b1-4a67-a7e7-d541cbd820d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:21.458793 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1f6da-57b1-4a67-a7e7-d541cbd820d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:21.460688 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1f6da-57b1-4a67-a7e7-d541cbd820d3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:21.473850 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:21.487905 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1350/2687] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:41:21 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:41:21.491972 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e86eb32-f5eb-4978-a193-f4c6bed6e79c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:21.502720 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e86eb32-f5eb-4978-a193-f4c6bed6e79c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:21.558816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e9d46-dfed-42d6-9434-425bb7dd6425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:21.559512 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e9d46-dfed-42d6-9434-425bb7dd6425 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:21.559794 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e9d46-dfed-42d6-9434-425bb7dd6425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:21.560104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e9d46-dfed-42d6-9434-425bb7dd6425 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:21.560799 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e9d46-dfed-42d6-9434-425bb7dd6425 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:21.562752 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e9d46-dfed-42d6-9434-425bb7dd6425 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:21.582064 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1338/2688] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:41:21 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:41:21.586030 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1f6da-57b1-4a67-a7e7-d541cbd820d3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:21.597166 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1f6da-57b1-4a67-a7e7-d541cbd820d3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:21.601751 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee0d9e-8398-45c9-b9c7-355d4c427b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:21.602408 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee0d9e-8398-45c9-b9c7-355d4c427b78 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:21.602726 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee0d9e-8398-45c9-b9c7-355d4c427b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:21.603012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee0d9e-8398-45c9-b9c7-355d4c427b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:21.603733 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee0d9e-8398-45c9-b9c7-355d4c427b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:21.605318 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee0d9e-8398-45c9-b9c7-355d4c427b78 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:21.616420 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1351/2689] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:41:21.620306 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e9d46-dfed-42d6-9434-425bb7dd6425 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:21.630575 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e9d46-dfed-42d6-9434-425bb7dd6425 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:21.635096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a4366b-84c6-473f-8139-5b7638cecc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:21.635776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4366b-84c6-473f-8139-5b7638cecc38 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:21.636062 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4366b-84c6-473f-8139-5b7638cecc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:21.636340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4366b-84c6-473f-8139-5b7638cecc38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:21.637057 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a4366b-84c6-473f-8139-5b7638cecc38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:21.638663 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a4366b-84c6-473f-8139-5b7638cecc38 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:21.649290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1339/2690] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:41:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:41:21.653273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee0d9e-8398-45c9-b9c7-355d4c427b78 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:21.664654 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee0d9e-8398-45c9-b9c7-355d4c427b78 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:21.669305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ce7d5-8d95-47d1-a76e-f6ea328ee9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:21.670190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ce7d5-8d95-47d1-a76e-f6ea328ee9f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:21.670508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ce7d5-8d95-47d1-a76e-f6ea328ee9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:21.670789 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ce7d5-8d95-47d1-a76e-f6ea328ee9f9 admin admin] PATH_INFO: `/v3/projects/de2500ed68ac42c0b6c53d342dfca4ef/users/f0aa942dfa8744d8bfe8f1128f079f29/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:21.685233 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ce7d5-8d95-47d1-a76e-f6ea328ee9f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de2500ed68ac42c0b6c53d342dfca4ef, user_id=f0aa942dfa8744d8bfe8f1128f079f29, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:21.687875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58ce7d5-8d95-47d1-a76e-f6ea328ee9f9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:21.700316 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1352/2691] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:41:21 2022] PUT /identity/v3/projects/de2500ed68ac42c0b6c53d342dfca4ef/users/f0aa942dfa8744d8bfe8f1128f079f29/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:41:21.712777 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1313050-df58-48ff-96a4-28b417e8f4e5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:21.712994 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1313050-df58-48ff-96a4-28b417e8f4e5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:21.713274 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1313050-df58-48ff-96a4-28b417e8f4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:21.740400 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b1313050-df58-48ff-96a4-28b417e8f4e5 None None] MFA Rules not processed for user `f0aa942dfa8744d8bfe8f1128f079f29`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:41:21.741599 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b1313050-df58-48ff-96a4-28b417e8f4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:21.805064 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1340/2692] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:41:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:41:21.809556 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ce7d5-8d95-47d1-a76e-f6ea328ee9f9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:21.869509 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ce7d5-8d95-47d1-a76e-f6ea328ee9f9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:21.874007 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49832e-e759-43c7-8ca3-f03aaba49300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:21.874688 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49832e-e759-43c7-8ca3-f03aaba49300 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:21.874970 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49832e-e759-43c7-8ca3-f03aaba49300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:21.875244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49832e-e759-43c7-8ca3-f03aaba49300 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:21.876075 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a49832e-e759-43c7-8ca3-f03aaba49300 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:21.877977 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a49832e-e759-43c7-8ca3-f03aaba49300 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:21.902853 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1353/2693] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:41:21 2022] POST /identity/v3/projects => generated 375 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:41:21.906975 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1313050-df58-48ff-96a4-28b417e8f4e5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:21.917780 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1313050-df58-48ff-96a4-28b417e8f4e5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:21.959416 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:21.976504 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888939e1-d69b-4656-8674-91654b8c5117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:21.977128 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888939e1-d69b-4656-8674-91654b8c5117 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:21.977408 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888939e1-d69b-4656-8674-91654b8c5117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:21.977681 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888939e1-d69b-4656-8674-91654b8c5117 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:21.978374 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888939e1-d69b-4656-8674-91654b8c5117 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:21.980303 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888939e1-d69b-4656-8674-91654b8c5117 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:22.001426 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1341/2694] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:41:21 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:41:22.005393 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49832e-e759-43c7-8ca3-f03aaba49300 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:22.015716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49832e-e759-43c7-8ca3-f03aaba49300 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:22.021026 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9103fd-70a3-405e-8f66-5371b7c2b362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:22.021679 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9103fd-70a3-405e-8f66-5371b7c2b362 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:22.021963 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9103fd-70a3-405e-8f66-5371b7c2b362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:22.022235 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9103fd-70a3-405e-8f66-5371b7c2b362 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:22.022999 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9103fd-70a3-405e-8f66-5371b7c2b362 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:22.024575 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9103fd-70a3-405e-8f66-5371b7c2b362 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:22.036230 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1354/2695] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:41:22.040254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888939e1-d69b-4656-8674-91654b8c5117 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:22.051469 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888939e1-d69b-4656-8674-91654b8c5117 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:22.057528 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3910be0-80db-44a1-bcff-b8a4468b1987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:22.058571 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3910be0-80db-44a1-bcff-b8a4468b1987 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:22.058889 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3910be0-80db-44a1-bcff-b8a4468b1987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:22.059173 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3910be0-80db-44a1-bcff-b8a4468b1987 admin admin] PATH_INFO: `/v3/projects/b634903412e24e909196d0082c4c2341/users/4f631c1874f546ffba189ae9c2a6bc98/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:22.074818 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3910be0-80db-44a1-bcff-b8a4468b1987 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b634903412e24e909196d0082c4c2341, user_id=4f631c1874f546ffba189ae9c2a6bc98, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:22.077508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3910be0-80db-44a1-bcff-b8a4468b1987 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:22.090678 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1342/2696] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:41:22 2022] PUT /identity/v3/projects/b634903412e24e909196d0082c4c2341/users/4f631c1874f546ffba189ae9c2a6bc98/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:22.095263 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9103fd-70a3-405e-8f66-5371b7c2b362 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:22.105397 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9103fd-70a3-405e-8f66-5371b7c2b362 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:22.162288 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4215ea-4f80-4405-971f-40455a17b58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:22.163033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4215ea-4f80-4405-971f-40455a17b58b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:22.163319 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4215ea-4f80-4405-971f-40455a17b58b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:22.163596 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4215ea-4f80-4405-971f-40455a17b58b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:22.164307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4215ea-4f80-4405-971f-40455a17b58b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:22.165893 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4215ea-4f80-4405-971f-40455a17b58b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:22.176035 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1355/2697] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:41:22 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:41:22.180323 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3910be0-80db-44a1-bcff-b8a4468b1987 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:22.190783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3910be0-80db-44a1-bcff-b8a4468b1987 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:22.195326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c6a1f-0029-4eb2-8bdd-1bdec4f4e569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:22.196200 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c6a1f-0029-4eb2-8bdd-1bdec4f4e569 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:22.196488 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c6a1f-0029-4eb2-8bdd-1bdec4f4e569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:22.196769 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8c6a1f-0029-4eb2-8bdd-1bdec4f4e569 admin admin] PATH_INFO: `/v3/projects/b634903412e24e909196d0082c4c2341/users/4f631c1874f546ffba189ae9c2a6bc98/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:22.198745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c6a1f-0029-4eb2-8bdd-1bdec4f4e569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b634903412e24e909196d0082c4c2341, user_id=4f631c1874f546ffba189ae9c2a6bc98, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:22.201360 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8c6a1f-0029-4eb2-8bdd-1bdec4f4e569 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:22.212138 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1343/2698] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:41:22 2022] PUT /identity/v3/projects/b634903412e24e909196d0082c4c2341/users/4f631c1874f546ffba189ae9c2a6bc98/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:22.222948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c24a1-56ab-4c0e-8c84-55242116efdb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:22.223258 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c24a1-56ab-4c0e-8c84-55242116efdb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:22.223524 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2c24a1-56ab-4c0e-8c84-55242116efdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:22.249353 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0c2c24a1-56ab-4c0e-8c84-55242116efdb None None] MFA Rules not processed for user `4f631c1874f546ffba189ae9c2a6bc98`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:41:22.250729 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0c2c24a1-56ab-4c0e-8c84-55242116efdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:22.266760 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:22.321280 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1356/2699] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:41:22 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:41:22.332720 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c6a1f-0029-4eb2-8bdd-1bdec4f4e569 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:22.334384 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bc8c6a1f-0029-4eb2-8bdd-1bdec4f4e569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:22.393664 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8c6a1f-0029-4eb2-8bdd-1bdec4f4e569 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:22.397598 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0017510-c178-4c73-ac9d-c537ea962e63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:22.398262 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0017510-c178-4c73-ac9d-c537ea962e63 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:22.398575 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0017510-c178-4c73-ac9d-c537ea962e63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:22.398872 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0017510-c178-4c73-ac9d-c537ea962e63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:22.454649 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0017510-c178-4c73-ac9d-c537ea962e63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:22.456100 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0017510-c178-4c73-ac9d-c537ea962e63 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:22.464866 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1344/2700] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:41:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:41:23.928927 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c24a1-56ab-4c0e-8c84-55242116efdb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:23.930305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0c2c24a1-56ab-4c0e-8c84-55242116efdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:23.943103 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2c24a1-56ab-4c0e-8c84-55242116efdb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:23.946581 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f238834-58c8-4eb5-9d81-2df34153ca77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:23.947222 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f238834-58c8-4eb5-9d81-2df34153ca77 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:23.947504 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f238834-58c8-4eb5-9d81-2df34153ca77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:23.947782 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f238834-58c8-4eb5-9d81-2df34153ca77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:24.003342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f238834-58c8-4eb5-9d81-2df34153ca77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:24.004765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f238834-58c8-4eb5-9d81-2df34153ca77 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:24.013477 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1357/2701] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:41:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:41:24.558171 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0017510-c178-4c73-ac9d-c537ea962e63 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:24.568021 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0017510-c178-4c73-ac9d-c537ea962e63 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:24.571693 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e01b3-878a-44f7-bc00-a5e8d8e211bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:24.572341 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e01b3-878a-44f7-bc00-a5e8d8e211bd service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:24.572631 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e01b3-878a-44f7-bc00-a5e8d8e211bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:24.572948 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e01b3-878a-44f7-bc00-a5e8d8e211bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:24.583429 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e01b3-878a-44f7-bc00-a5e8d8e211bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:24.584870 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e01b3-878a-44f7-bc00-a5e8d8e211bd service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:24.594791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1345/2702] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:41:24 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:41:28.422691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f238834-58c8-4eb5-9d81-2df34153ca77 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:28.487004 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f238834-58c8-4eb5-9d81-2df34153ca77 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:28.491543 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40559ee1-46fa-4abf-8f94-714a30da22a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:28.492401 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40559ee1-46fa-4abf-8f94-714a30da22a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:28.492694 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40559ee1-46fa-4abf-8f94-714a30da22a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:28.492970 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40559ee1-46fa-4abf-8f94-714a30da22a3 admin admin] PATH_INFO: `/v3/users/0fb1508d9f874110a672d5a1f27d64d1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:28.494141 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40559ee1-46fa-4abf-8f94-714a30da22a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fb1508d9f874110a672d5a1f27d64d1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:28.496087 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40559ee1-46fa-4abf-8f94-714a30da22a3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:28.543720 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-40559ee1-46fa-4abf-8f94-714a30da22a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fb1508d9f874110a672d5a1f27d64d1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:28.544376 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-40559ee1-46fa-4abf-8f94-714a30da22a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fb1508d9f874110a672d5a1f27d64d1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:28.550038 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:28.558073 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-40559ee1-46fa-4abf-8f94-714a30da22a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fb1508d9f874110a672d5a1f27d64d1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:28.558751 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-40559ee1-46fa-4abf-8f94-714a30da22a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fb1508d9f874110a672d5a1f27d64d1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:28.566143 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-40559ee1-46fa-4abf-8f94-714a30da22a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fb1508d9f874110a672d5a1f27d64d1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:28.579078 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1358/2703] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:41:28 2022] DELETE /identity/v3/users/0fb1508d9f874110a672d5a1f27d64d1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:28.583625 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3abcab-95a1-4b70-ad8c-c12d3a128e46 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:28.584104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3abcab-95a1-4b70-ad8c-c12d3a128e46 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:28.584221 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3abcab-95a1-4b70-ad8c-c12d3a128e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:28.601267 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ca3abcab-95a1-4b70-ad8c-c12d3a128e46 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:41:28.602539 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ca3abcab-95a1-4b70-ad8c-c12d3a128e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:28.631507 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:28.661429 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1346/2704] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:41:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:41:28.671269 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40559ee1-46fa-4abf-8f94-714a30da22a3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:28.672730 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-40559ee1-46fa-4abf-8f94-714a30da22a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:28.729687 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40559ee1-46fa-4abf-8f94-714a30da22a3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:28.733296 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912f384-fb21-45ba-8109-c66ea8cea722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:28.733949 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912f384-fb21-45ba-8109-c66ea8cea722 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:28.734250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912f384-fb21-45ba-8109-c66ea8cea722 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:28.734555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f912f384-fb21-45ba-8109-c66ea8cea722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:28.743887 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f912f384-fb21-45ba-8109-c66ea8cea722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:28.745303 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f912f384-fb21-45ba-8109-c66ea8cea722 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:28.755133 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1359/2705] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:41:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:41:29.030174 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3abcab-95a1-4b70-ad8c-c12d3a128e46 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:29.032127 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ca3abcab-95a1-4b70-ad8c-c12d3a128e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:29.045142 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3abcab-95a1-4b70-ad8c-c12d3a128e46 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:29.050562 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902a969-5e16-46b9-824f-744bec7624de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:29.051433 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902a969-5e16-46b9-824f-744bec7624de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:29.051719 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902a969-5e16-46b9-824f-744bec7624de admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:29.051995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902a969-5e16-46b9-824f-744bec7624de admin admin] PATH_INFO: `/v3/projects/6fa35cc45e864e6b898e72ad724cad82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:29.053169 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b902a969-5e16-46b9-824f-744bec7624de admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fa35cc45e864e6b898e72ad724cad82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:29.055178 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b902a969-5e16-46b9-824f-744bec7624de admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:29.102831 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b902a969-5e16-46b9-824f-744bec7624de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fa35cc45e864e6b898e72ad724cad82', 'request_id': 'req-b902a969-5e16-46b9-824f-744bec7624de'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:29.108785 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b902a969-5e16-46b9-824f-744bec7624de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fa35cc45e864e6b898e72ad724cad82', 'request_id': 'req-b902a969-5e16-46b9-824f-744bec7624de'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:29.115165 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b902a969-5e16-46b9-824f-744bec7624de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fa35cc45e864e6b898e72ad724cad82', 'request_id': 'req-b902a969-5e16-46b9-824f-744bec7624de'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:29.127350 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b902a969-5e16-46b9-824f-744bec7624de admin admin] The token cache is being invalidate because project 6fa35cc45e864e6b898e72ad724cad82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:41:29.127642 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b902a969-5e16-46b9-824f-744bec7624de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:29.128254 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b902a969-5e16-46b9-824f-744bec7624de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:29.129783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1347/2706] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:41:29 2022] DELETE /identity/v3/projects/6fa35cc45e864e6b898e72ad724cad82 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:29.133628 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912f384-fb21-45ba-8109-c66ea8cea722 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:29.144008 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f912f384-fb21-45ba-8109-c66ea8cea722 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:29.201440 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49409a-c0f8-4145-896d-af86c4d4cc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:29.202299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49409a-c0f8-4145-896d-af86c4d4cc16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:29.202613 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49409a-c0f8-4145-896d-af86c4d4cc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:29.202889 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49409a-c0f8-4145-896d-af86c4d4cc16 admin admin] PATH_INFO: `/v3/users/f0aa942dfa8744d8bfe8f1128f079f29` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:29.204748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f49409a-c0f8-4145-896d-af86c4d4cc16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0aa942dfa8744d8bfe8f1128f079f29)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:29.206804 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f49409a-c0f8-4145-896d-af86c4d4cc16 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:29.253068 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8f49409a-c0f8-4145-896d-af86c4d4cc16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0aa942dfa8744d8bfe8f1128f079f29'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:29.253864 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8f49409a-c0f8-4145-896d-af86c4d4cc16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0aa942dfa8744d8bfe8f1128f079f29'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:29.266551 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8f49409a-c0f8-4145-896d-af86c4d4cc16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0aa942dfa8744d8bfe8f1128f079f29'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:29.267535 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8f49409a-c0f8-4145-896d-af86c4d4cc16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0aa942dfa8744d8bfe8f1128f079f29'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:29.274781 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8f49409a-c0f8-4145-896d-af86c4d4cc16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0aa942dfa8744d8bfe8f1128f079f29'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:29.277422 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:29.290143 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1360/2707] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:41:29 2022] DELETE /identity/v3/users/f0aa942dfa8744d8bfe8f1128f079f29 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:29.554619 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902a969-5e16-46b9-824f-744bec7624de admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:29.556124 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b902a969-5e16-46b9-824f-744bec7624de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:29.571749 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902a969-5e16-46b9-824f-744bec7624de admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:29.629869 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbb731-6e04-4413-ac09-05b54541e747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:29.630670 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbb731-6e04-4413-ac09-05b54541e747 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:29.630952 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbb731-6e04-4413-ac09-05b54541e747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:29.631228 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbb731-6e04-4413-ac09-05b54541e747 admin admin] PATH_INFO: `/v3/projects/de2500ed68ac42c0b6c53d342dfca4ef` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:29.632380 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dbb731-6e04-4413-ac09-05b54541e747 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de2500ed68ac42c0b6c53d342dfca4ef)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:29.634262 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dbb731-6e04-4413-ac09-05b54541e747 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:29.678137 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f5dbb731-6e04-4413-ac09-05b54541e747 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de2500ed68ac42c0b6c53d342dfca4ef', 'request_id': 'req-f5dbb731-6e04-4413-ac09-05b54541e747'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:29.684001 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f5dbb731-6e04-4413-ac09-05b54541e747 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de2500ed68ac42c0b6c53d342dfca4ef', 'request_id': 'req-f5dbb731-6e04-4413-ac09-05b54541e747'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:29.689003 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:29.690237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f5dbb731-6e04-4413-ac09-05b54541e747 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de2500ed68ac42c0b6c53d342dfca4ef', 'request_id': 'req-f5dbb731-6e04-4413-ac09-05b54541e747'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:29.706222 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f5dbb731-6e04-4413-ac09-05b54541e747 admin admin] The token cache is being invalidate because project de2500ed68ac42c0b6c53d342dfca4ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:41:29.706544 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f5dbb731-6e04-4413-ac09-05b54541e747 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:29.707162 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f5dbb731-6e04-4413-ac09-05b54541e747 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:29.708691 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1348/2708] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:41:29 2022] DELETE /identity/v3/projects/de2500ed68ac42c0b6c53d342dfca4ef => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:29.712826 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49409a-c0f8-4145-896d-af86c4d4cc16 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:29.714385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8f49409a-c0f8-4145-896d-af86c4d4cc16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:29.728322 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49409a-c0f8-4145-896d-af86c4d4cc16 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:29.786658 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0442a2-6605-49b2-ba81-bde8881dcc86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:29.787544 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0442a2-6605-49b2-ba81-bde8881dcc86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:29.787832 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0442a2-6605-49b2-ba81-bde8881dcc86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:29.788112 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0442a2-6605-49b2-ba81-bde8881dcc86 admin admin] PATH_INFO: `/v3/users/4f631c1874f546ffba189ae9c2a6bc98` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:29.789305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0442a2-6605-49b2-ba81-bde8881dcc86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f631c1874f546ffba189ae9c2a6bc98)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:29.791262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0442a2-6605-49b2-ba81-bde8881dcc86 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:29.837078 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ea0442a2-6605-49b2-ba81-bde8881dcc86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f631c1874f546ffba189ae9c2a6bc98'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:29.837710 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ea0442a2-6605-49b2-ba81-bde8881dcc86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f631c1874f546ffba189ae9c2a6bc98'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:29.849449 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ea0442a2-6605-49b2-ba81-bde8881dcc86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f631c1874f546ffba189ae9c2a6bc98'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:29.850064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ea0442a2-6605-49b2-ba81-bde8881dcc86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f631c1874f546ffba189ae9c2a6bc98'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:29.856876 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ea0442a2-6605-49b2-ba81-bde8881dcc86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f631c1874f546ffba189ae9c2a6bc98'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:29.869638 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1361/2709] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:41:29 2022] DELETE /identity/v3/users/4f631c1874f546ffba189ae9c2a6bc98 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:30.148064 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbb731-6e04-4413-ac09-05b54541e747 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:30.149502 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f5dbb731-6e04-4413-ac09-05b54541e747 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:30.163823 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbb731-6e04-4413-ac09-05b54541e747 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:30.222873 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ffa28-8606-4b8c-9acf-83702bf06290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:30.223730 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ffa28-8606-4b8c-9acf-83702bf06290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:30.224012 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ffa28-8606-4b8c-9acf-83702bf06290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:30.224287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ffa28-8606-4b8c-9acf-83702bf06290 admin admin] PATH_INFO: `/v3/projects/b634903412e24e909196d0082c4c2341` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:30.225450 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ffa28-8606-4b8c-9acf-83702bf06290 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b634903412e24e909196d0082c4c2341)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:30.227395 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ffa28-8606-4b8c-9acf-83702bf06290 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:30.279157 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ea9ffa28-8606-4b8c-9acf-83702bf06290 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b634903412e24e909196d0082c4c2341', 'request_id': 'req-ea9ffa28-8606-4b8c-9acf-83702bf06290'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:30.286436 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ea9ffa28-8606-4b8c-9acf-83702bf06290 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b634903412e24e909196d0082c4c2341', 'request_id': 'req-ea9ffa28-8606-4b8c-9acf-83702bf06290'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:30.292834 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ea9ffa28-8606-4b8c-9acf-83702bf06290 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b634903412e24e909196d0082c4c2341', 'request_id': 'req-ea9ffa28-8606-4b8c-9acf-83702bf06290'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:30.307711 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ea9ffa28-8606-4b8c-9acf-83702bf06290 admin admin] The token cache is being invalidate because project b634903412e24e909196d0082c4c2341 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:41:30.308204 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ea9ffa28-8606-4b8c-9acf-83702bf06290 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:30.309200 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ea9ffa28-8606-4b8c-9acf-83702bf06290 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:41:30.312469 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1349/2710] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:41:30 2022] DELETE /identity/v3/projects/b634903412e24e909196d0082c4c2341 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:30.319593 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483bcca-a2d2-4eff-87a1-6c32358bf5c5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:30.319980 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483bcca-a2d2-4eff-87a1-6c32358bf5c5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:30.320333 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483bcca-a2d2-4eff-87a1-6c32358bf5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:30.341694 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1483bcca-a2d2-4eff-87a1-6c32358bf5c5 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:41:30.343061 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1483bcca-a2d2-4eff-87a1-6c32358bf5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:30.404543 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1362/2711] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:41:30.412575 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d6661-2d8c-456b-90ce-b82cd60fc7a5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:30.412862 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d6661-2d8c-456b-90ce-b82cd60fc7a5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:30.413142 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d6661-2d8c-456b-90ce-b82cd60fc7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:30.431604 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f17d6661-2d8c-456b-90ce-b82cd60fc7a5 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:41:30.432805 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f17d6661-2d8c-456b-90ce-b82cd60fc7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:30.441403 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1350/2712] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:41:30.445491 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1483bcca-a2d2-4eff-87a1-6c32358bf5c5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:30.455843 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1483bcca-a2d2-4eff-87a1-6c32358bf5c5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:30.460450 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7cf400-53d5-4c40-9fca-e6f5cdd55123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:30.461105 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7cf400-53d5-4c40-9fca-e6f5cdd55123 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:30.461395 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7cf400-53d5-4c40-9fca-e6f5cdd55123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:30.461676 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7cf400-53d5-4c40-9fca-e6f5cdd55123 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:30.462403 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7cf400-53d5-4c40-9fca-e6f5cdd55123 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:41:30.462747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7cf400-53d5-4c40-9fca-e6f5cdd55123 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:30.464336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7cf400-53d5-4c40-9fca-e6f5cdd55123 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:30.472944 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:30.476569 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1363/2713] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:41:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:41:30.480607 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d6661-2d8c-456b-90ce-b82cd60fc7a5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:30.489813 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d6661-2d8c-456b-90ce-b82cd60fc7a5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:30.494530 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb46aa5-c35e-4290-b3b5-dd12a6616daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:30.495173 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb46aa5-c35e-4290-b3b5-dd12a6616daa admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:30.495458 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb46aa5-c35e-4290-b3b5-dd12a6616daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:30.495740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb46aa5-c35e-4290-b3b5-dd12a6616daa admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:30.496428 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb46aa5-c35e-4290-b3b5-dd12a6616daa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:30.498321 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb46aa5-c35e-4290-b3b5-dd12a6616daa admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:30.524531 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1351/2714] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:41:30 2022] POST /identity/v3/projects => generated 365 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:41:30.528774 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7cf400-53d5-4c40-9fca-e6f5cdd55123 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:30.539391 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7cf400-53d5-4c40-9fca-e6f5cdd55123 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:30.607398 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bf4c7-cbd7-463e-9fb5-0a2ca3ba8346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:30.607398 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bf4c7-cbd7-463e-9fb5-0a2ca3ba8346 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:30.607398 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bf4c7-cbd7-463e-9fb5-0a2ca3ba8346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:30.607810 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831bf4c7-cbd7-463e-9fb5-0a2ca3ba8346 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:30.608281 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831bf4c7-cbd7-463e-9fb5-0a2ca3ba8346 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:30.610334 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831bf4c7-cbd7-463e-9fb5-0a2ca3ba8346 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:30.636026 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1364/2715] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:41:30 2022] POST /identity/v3/users => generated 283 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:41:30.640401 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb46aa5-c35e-4290-b3b5-dd12a6616daa admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:30.653447 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb46aa5-c35e-4290-b3b5-dd12a6616daa admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:30.659432 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc90d99-0231-44bd-a4b3-19833b6df3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:30.660211 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc90d99-0231-44bd-a4b3-19833b6df3ac admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:30.660523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc90d99-0231-44bd-a4b3-19833b6df3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:30.660811 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc90d99-0231-44bd-a4b3-19833b6df3ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:30.661576 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc90d99-0231-44bd-a4b3-19833b6df3ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:30.663268 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc90d99-0231-44bd-a4b3-19833b6df3ac admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:30.676477 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1352/2716] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:41:30.681045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bf4c7-cbd7-463e-9fb5-0a2ca3ba8346 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:30.692520 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831bf4c7-cbd7-463e-9fb5-0a2ca3ba8346 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:30.697795 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd3d56-c4ec-4f8b-a1e5-a77e2651ee59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:30.698530 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd3d56-c4ec-4f8b-a1e5-a77e2651ee59 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:30.699031 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd3d56-c4ec-4f8b-a1e5-a77e2651ee59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:30.699318 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd3d56-c4ec-4f8b-a1e5-a77e2651ee59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:30.700575 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd3d56-c4ec-4f8b-a1e5-a77e2651ee59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:30.702583 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dd3d56-c4ec-4f8b-a1e5-a77e2651ee59 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:30.713919 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1365/2717] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:41:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:41:30.718297 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc90d99-0231-44bd-a4b3-19833b6df3ac admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:30.728964 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc90d99-0231-44bd-a4b3-19833b6df3ac admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:30.735565 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4deed7-eee1-4cca-b733-11ff12503ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:30.737864 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4deed7-eee1-4cca-b733-11ff12503ed1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:30.737864 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4deed7-eee1-4cca-b733-11ff12503ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:30.737864 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4deed7-eee1-4cca-b733-11ff12503ed1 admin admin] PATH_INFO: `/v3/projects/9252260ef37347bdaa27a5cd518d8305/users/aa9b9cd419844e22a7e8fe99cde8963c/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:30.782509 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4deed7-eee1-4cca-b733-11ff12503ed1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9252260ef37347bdaa27a5cd518d8305, user_id=aa9b9cd419844e22a7e8fe99cde8963c, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:30.785788 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4deed7-eee1-4cca-b733-11ff12503ed1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:30.804034 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1353/2718] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:41:30 2022] PUT /identity/v3/projects/9252260ef37347bdaa27a5cd518d8305/users/aa9b9cd419844e22a7e8fe99cde8963c/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:41:30.816768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c02b2d-01ee-4fdc-8e5a-bb5f180d315e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:30.817071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c02b2d-01ee-4fdc-8e5a-bb5f180d315e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:30.817357 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c02b2d-01ee-4fdc-8e5a-bb5f180d315e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:30.844990 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-23c02b2d-01ee-4fdc-8e5a-bb5f180d315e None None] MFA Rules not processed for user `aa9b9cd419844e22a7e8fe99cde8963c`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:41:30.846220 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-23c02b2d-01ee-4fdc-8e5a-bb5f180d315e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:30.923567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1366/2719] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:41:30 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:41:30.935371 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4deed7-eee1-4cca-b733-11ff12503ed1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:30.937064 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-eb4deed7-eee1-4cca-b733-11ff12503ed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:30.957523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:31.002505 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4deed7-eee1-4cca-b733-11ff12503ed1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:31.006122 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ff6eb-bb68-41ad-b300-dfe035bda47f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:31.006799 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ff6eb-bb68-41ad-b300-dfe035bda47f service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:31.007088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ff6eb-bb68-41ad-b300-dfe035bda47f service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:31.007370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ff6eb-bb68-41ad-b300-dfe035bda47f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:31.016042 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45ff6eb-bb68-41ad-b300-dfe035bda47f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:31.018301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45ff6eb-bb68-41ad-b300-dfe035bda47f service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:31.027394 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1354/2720] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:41:30 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:41:35.015099 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c02b2d-01ee-4fdc-8e5a-bb5f180d315e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:35.016893 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-23c02b2d-01ee-4fdc-8e5a-bb5f180d315e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:35.122648 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c02b2d-01ee-4fdc-8e5a-bb5f180d315e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:35.132382 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54620d-b7e8-41eb-bb76-3cb722a965bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98135cacb9d4f22846ad33500b9bd1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:35.133083 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54620d-b7e8-41eb-bb76-3cb722a965bf service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:35.133494 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54620d-b7e8-41eb-bb76-3cb722a965bf service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:35.133772 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54620d-b7e8-41eb-bb76-3cb722a965bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:35.138080 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9b54620d-b7e8-41eb-bb76-3cb722a965bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:35.160321 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54620d-b7e8-41eb-bb76-3cb722a965bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:35.162716 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54620d-b7e8-41eb-bb76-3cb722a965bf service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:35.168049 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:35.177674 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1367/2721] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:41:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:41:37.920390 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69454c18-c35a-47ba-9a31-bc1875319fa3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:37.920942 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69454c18-c35a-47ba-9a31-bc1875319fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:37.921037 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69454c18-c35a-47ba-9a31-bc1875319fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:37.938297 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-69454c18-c35a-47ba-9a31-bc1875319fa3 None None] MFA Rules not processed for user `2af4d4a69e9d4755bff61401f4349c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:41:37.939552 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-69454c18-c35a-47ba-9a31-bc1875319fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:37.994822 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1355/2722] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:41:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:41:38.296724 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54620d-b7e8-41eb-bb76-3cb722a965bf service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:41:38.299267 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9b54620d-b7e8-41eb-bb76-3cb722a965bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:38.356362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54620d-b7e8-41eb-bb76-3cb722a965bf service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:41:38.360156 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f3f1d6-47e8-4768-9b40-9986f30ca4f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:41:38.360834 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3f1d6-47e8-4768-9b40-9986f30ca4f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:41:38.361118 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3f1d6-47e8-4768-9b40-9986f30ca4f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:41:38.361393 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3f1d6-47e8-4768-9b40-9986f30ca4f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:41:38.363269 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-25f3f1d6-47e8-4768-9b40-9986f30ca4f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:41:38.372959 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3f1d6-47e8-4768-9b40-9986f30ca4f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:41:38.374373 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f3f1d6-47e8-4768-9b40-9986f30ca4f1 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:41:38.383064 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1368/2723] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:41:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:41:47.988201 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=44 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:41:48.379482 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=27 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:42:04.940006 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69454c18-c35a-47ba-9a31-bc1875319fa3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:42:05.004409 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69454c18-c35a-47ba-9a31-bc1875319fa3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:42:05.008897 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffeb95-74d3-4edd-a201-debaac4cf82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:42:05.009764 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffeb95-74d3-4edd-a201-debaac4cf82a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:42:05.010050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffeb95-74d3-4edd-a201-debaac4cf82a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:42:05.010333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffeb95-74d3-4edd-a201-debaac4cf82a admin admin] PATH_INFO: `/v3/users/aa9b9cd419844e22a7e8fe99cde8963c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:42:05.011548 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffeb95-74d3-4edd-a201-debaac4cf82a admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa9b9cd419844e22a7e8fe99cde8963c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:42:05.013528 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffeb95-74d3-4edd-a201-debaac4cf82a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:42:05.060836 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c0ffeb95-74d3-4edd-a201-debaac4cf82a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa9b9cd419844e22a7e8fe99cde8963c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:42:05.068256 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c0ffeb95-74d3-4edd-a201-debaac4cf82a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa9b9cd419844e22a7e8fe99cde8963c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:42:05.080662 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c0ffeb95-74d3-4edd-a201-debaac4cf82a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa9b9cd419844e22a7e8fe99cde8963c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:42:05.081306 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c0ffeb95-74d3-4edd-a201-debaac4cf82a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa9b9cd419844e22a7e8fe99cde8963c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:42:05.081949 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c0ffeb95-74d3-4edd-a201-debaac4cf82a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa9b9cd419844e22a7e8fe99cde8963c'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:42:05.094495 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1356/2724] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:42:04 2022] DELETE /identity/v3/users/aa9b9cd419844e22a7e8fe99cde8963c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:42:05.099632 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328e614-f77b-46d8-be79-f090b27462c7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:42:05.100118 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328e614-f77b-46d8-be79-f090b27462c7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:42:05.100235 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328e614-f77b-46d8-be79-f090b27462c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:42:05.118215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8328e614-f77b-46d8-be79-f090b27462c7 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:42:05.119439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8328e614-f77b-46d8-be79-f090b27462c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:42:05.176366 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1369/2725] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:42:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:42:05.185959 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffeb95-74d3-4edd-a201-debaac4cf82a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:42:05.187508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c0ffeb95-74d3-4edd-a201-debaac4cf82a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:42:05.244402 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffeb95-74d3-4edd-a201-debaac4cf82a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:42:05.248061 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b620deb-4af6-4d26-ae4f-89f316b4ffa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:42:05.248705 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b620deb-4af6-4d26-ae4f-89f316b4ffa4 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:42:05.248993 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b620deb-4af6-4d26-ae4f-89f316b4ffa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:42:05.249276 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b620deb-4af6-4d26-ae4f-89f316b4ffa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:42:05.258238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b620deb-4af6-4d26-ae4f-89f316b4ffa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:42:05.259713 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b620deb-4af6-4d26-ae4f-89f316b4ffa4 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:42:05.269752 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1357/2726] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:42:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:42:05.532153 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328e614-f77b-46d8-be79-f090b27462c7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:42:05.533719 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8328e614-f77b-46d8-be79-f090b27462c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:42:05.546715 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328e614-f77b-46d8-be79-f090b27462c7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:42:05.551466 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3754ddef-a537-40d6-9156-1ee36b4a8ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:42:05.552412 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754ddef-a537-40d6-9156-1ee36b4a8ce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:42:05.552768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754ddef-a537-40d6-9156-1ee36b4a8ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:42:05.553135 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3754ddef-a537-40d6-9156-1ee36b4a8ce7 admin admin] PATH_INFO: `/v3/projects/9252260ef37347bdaa27a5cd518d8305` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:42:05.554384 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754ddef-a537-40d6-9156-1ee36b4a8ce7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9252260ef37347bdaa27a5cd518d8305)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:42:05.556493 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3754ddef-a537-40d6-9156-1ee36b4a8ce7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:42:05.606131 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3754ddef-a537-40d6-9156-1ee36b4a8ce7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9252260ef37347bdaa27a5cd518d8305', 'request_id': 'req-3754ddef-a537-40d6-9156-1ee36b4a8ce7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:42:05.613772 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3754ddef-a537-40d6-9156-1ee36b4a8ce7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9252260ef37347bdaa27a5cd518d8305', 'request_id': 'req-3754ddef-a537-40d6-9156-1ee36b4a8ce7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:42:05.620115 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3754ddef-a537-40d6-9156-1ee36b4a8ce7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9252260ef37347bdaa27a5cd518d8305', 'request_id': 'req-3754ddef-a537-40d6-9156-1ee36b4a8ce7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:42:05.630750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3754ddef-a537-40d6-9156-1ee36b4a8ce7 admin admin] The token cache is being invalidate because project 9252260ef37347bdaa27a5cd518d8305 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:42:05.631037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3754ddef-a537-40d6-9156-1ee36b4a8ce7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:42:05.631658 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3754ddef-a537-40d6-9156-1ee36b4a8ce7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:42:05.633172 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1370/2727] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:42:05 2022] DELETE /identity/v3/projects/9252260ef37347bdaa27a5cd518d8305 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:42:05.638231 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f215c-44ac-44dd-bd99-391654e58cc9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:42:05.638556 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f215c-44ac-44dd-bd99-391654e58cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:42:05.638839 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f215c-44ac-44dd-bd99-391654e58cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:42:05.656502 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3d7f215c-44ac-44dd-bd99-391654e58cc9 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:42:05.657681 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3d7f215c-44ac-44dd-bd99-391654e58cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:42:05.674369 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:42:05.718331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1358/2728] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:42:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:42:05.726400 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5d830-3a9a-4a01-bf65-d7a82c782c66 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:42:05.726752 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5d830-3a9a-4a01-bf65-d7a82c782c66 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:42:05.727034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5d830-3a9a-4a01-bf65-d7a82c782c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:42:05.744968 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-baf5d830-3a9a-4a01-bf65-d7a82c782c66 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:42:05.746176 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-baf5d830-3a9a-4a01-bf65-d7a82c782c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:42:05.754963 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1371/2729] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:42:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:42:05.759227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f215c-44ac-44dd-bd99-391654e58cc9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:42:05.769119 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f215c-44ac-44dd-bd99-391654e58cc9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:42:05.774614 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44191649-0fbf-4d12-b03b-07d025ec7a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:42:05.775274 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44191649-0fbf-4d12-b03b-07d025ec7a8b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:42:05.775561 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44191649-0fbf-4d12-b03b-07d025ec7a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:42:05.775843 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44191649-0fbf-4d12-b03b-07d025ec7a8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:42:05.776568 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44191649-0fbf-4d12-b03b-07d025ec7a8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:42:05.776880 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44191649-0fbf-4d12-b03b-07d025ec7a8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:42:05.778499 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44191649-0fbf-4d12-b03b-07d025ec7a8b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:42:05.791572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1359/2730] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:42:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:42:05.795777 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5d830-3a9a-4a01-bf65-d7a82c782c66 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:42:05.806385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5d830-3a9a-4a01-bf65-d7a82c782c66 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:42:05.811039 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1207fc-8c28-492e-ab50-e619a4dc00e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:42:05.811703 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1207fc-8c28-492e-ab50-e619a4dc00e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:42:05.812039 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1207fc-8c28-492e-ab50-e619a4dc00e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:42:05.812317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1207fc-8c28-492e-ab50-e619a4dc00e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:42:05.813048 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1207fc-8c28-492e-ab50-e619a4dc00e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:42:05.815072 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1207fc-8c28-492e-ab50-e619a4dc00e6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:42:05.841688 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1372/2731] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:42:05 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:42:05.845835 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44191649-0fbf-4d12-b03b-07d025ec7a8b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:42:05.856744 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44191649-0fbf-4d12-b03b-07d025ec7a8b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:42:05.916295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b496613-6d73-4f72-8d0e-d87de06af4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:42:05.916969 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b496613-6d73-4f72-8d0e-d87de06af4ed admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:42:05.917256 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b496613-6d73-4f72-8d0e-d87de06af4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:42:05.917568 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b496613-6d73-4f72-8d0e-d87de06af4ed admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:42:05.918271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b496613-6d73-4f72-8d0e-d87de06af4ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:42:05.920218 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b496613-6d73-4f72-8d0e-d87de06af4ed admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:42:05.939851 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1360/2732] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:42:05 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:42:05.944307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1207fc-8c28-492e-ab50-e619a4dc00e6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:42:05.954598 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1207fc-8c28-492e-ab50-e619a4dc00e6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:42:05.959702 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ceb8f-ba65-4f8b-8acc-8ae5ba92de26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:42:05.960361 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ceb8f-ba65-4f8b-8acc-8ae5ba92de26 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:42:05.960646 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ceb8f-ba65-4f8b-8acc-8ae5ba92de26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:42:05.960928 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ceb8f-ba65-4f8b-8acc-8ae5ba92de26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:42:05.961661 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ceb8f-ba65-4f8b-8acc-8ae5ba92de26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:42:05.963288 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ceb8f-ba65-4f8b-8acc-8ae5ba92de26 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:42:05.974183 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1373/2733] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:42:05.977993 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b496613-6d73-4f72-8d0e-d87de06af4ed admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:42:05.988178 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b496613-6d73-4f72-8d0e-d87de06af4ed admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:42:05.992566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3985008-dd5c-4362-880a-219d047171e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:42:05.993214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3985008-dd5c-4362-880a-219d047171e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:42:05.993499 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3985008-dd5c-4362-880a-219d047171e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:42:05.993772 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3985008-dd5c-4362-880a-219d047171e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:42:05.994512 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3985008-dd5c-4362-880a-219d047171e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:42:05.996104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3985008-dd5c-4362-880a-219d047171e6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:42:06.005823 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1361/2734] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:42:05 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:42:06.009676 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ceb8f-ba65-4f8b-8acc-8ae5ba92de26 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:42:06.021710 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ceb8f-ba65-4f8b-8acc-8ae5ba92de26 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:42:06.026253 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658d2ff-cda7-47d0-9562-229be4589631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:42:06.027167 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658d2ff-cda7-47d0-9562-229be4589631 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:42:06.027460 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658d2ff-cda7-47d0-9562-229be4589631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:42:06.027740 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658d2ff-cda7-47d0-9562-229be4589631 admin admin] PATH_INFO: `/v3/projects/e1ba9e59fe574391a0c8342daed01416/users/eb9bae5e3522437182d2650c3fcfdef6/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:42:06.042993 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658d2ff-cda7-47d0-9562-229be4589631 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1ba9e59fe574391a0c8342daed01416, user_id=eb9bae5e3522437182d2650c3fcfdef6, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:42:06.045620 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658d2ff-cda7-47d0-9562-229be4589631 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:42:06.059328 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1374/2735] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:42:06 2022] PUT /identity/v3/projects/e1ba9e59fe574391a0c8342daed01416/users/eb9bae5e3522437182d2650c3fcfdef6/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:42:06.064602 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012211e7-98aa-4880-8ab6-d91d2b169cc2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:42:06.064814 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012211e7-98aa-4880-8ab6-d91d2b169cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:42:06.065115 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012211e7-98aa-4880-8ab6-d91d2b169cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:42:06.083398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-012211e7-98aa-4880-8ab6-d91d2b169cc2 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:42:06.084642 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-012211e7-98aa-4880-8ab6-d91d2b169cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:42:06.142810 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1362/2736] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:42:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:42:06.152735 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658d2ff-cda7-47d0-9562-229be4589631 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:42:06.154378 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2658d2ff-cda7-47d0-9562-229be4589631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:42:06.176567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:42:06.216894 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658d2ff-cda7-47d0-9562-229be4589631 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:42:06.220612 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ebd398-39da-45a1-a7c5-a9a8287125b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:42:06.221263 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebd398-39da-45a1-a7c5-a9a8287125b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:42:06.221552 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebd398-39da-45a1-a7c5-a9a8287125b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:42:06.221828 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ebd398-39da-45a1-a7c5-a9a8287125b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:42:06.232186 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ebd398-39da-45a1-a7c5-a9a8287125b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:42:06.233555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ebd398-39da-45a1-a7c5-a9a8287125b0 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:42:06.245668 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1375/2737] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:42:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:42:06.842289 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f4609-a350-468f-8abe-d7f6c9958101 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:42:06.842613 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f4609-a350-468f-8abe-d7f6c9958101 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:42:06.842903 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f4609-a350-468f-8abe-d7f6c9958101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:42:06.869902 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-821f4609-a350-468f-8abe-d7f6c9958101 None None] MFA Rules not processed for user `eb9bae5e3522437182d2650c3fcfdef6`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:42:06.871136 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-821f4609-a350-468f-8abe-d7f6c9958101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:42:06.888937 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:42:06.943295 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1363/2738] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:42:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:42:06.953872 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ebd398-39da-45a1-a7c5-a9a8287125b0 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:42:06.955291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-45ebd398-39da-45a1-a7c5-a9a8287125b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:42:07.015643 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ebd398-39da-45a1-a7c5-a9a8287125b0 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:42:07.020011 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f932af-9a96-4265-ba71-8a67e71170a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:42:07.020870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f932af-9a96-4265-ba71-8a67e71170a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:42:07.021157 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f932af-9a96-4265-ba71-8a67e71170a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:42:07.021431 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f932af-9a96-4265-ba71-8a67e71170a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:42:07.030930 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f932af-9a96-4265-ba71-8a67e71170a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:42:07.032403 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f932af-9a96-4265-ba71-8a67e71170a4 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:42:07.042741 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1376/2739] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:42:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:42:08.391657 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334f267-0805-45d2-85b1-07d090f20360 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:42:08.392139 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334f267-0805-45d2-85b1-07d090f20360 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:42:08.392257 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334f267-0805-45d2-85b1-07d090f20360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:42:08.410744 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0334f267-0805-45d2-85b1-07d090f20360 None None] MFA Rules not processed for user `eb9bae5e3522437182d2650c3fcfdef6`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:42:08.411926 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0334f267-0805-45d2-85b1-07d090f20360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:42:08.418625 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1364/2740] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:42:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:42:17.038707 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=43 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:42:18.404297 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=22 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:43:01.469962 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f932af-9a96-4265-ba71-8a67e71170a4 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:01.471593 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-72f932af-9a96-4265-ba71-8a67e71170a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:43:01.487856 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f932af-9a96-4265-ba71-8a67e71170a4 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:01.493094 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fad58a-fa39-4dc0-b856-6b3ad8249f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:01.493991 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fad58a-fa39-4dc0-b856-6b3ad8249f9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:01.494275 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fad58a-fa39-4dc0-b856-6b3ad8249f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:01.494590 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fad58a-fa39-4dc0-b856-6b3ad8249f9d admin admin] PATH_INFO: `/v3/users/2af4d4a69e9d4755bff61401f4349c78` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:01.495808 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fad58a-fa39-4dc0-b856-6b3ad8249f9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2af4d4a69e9d4755bff61401f4349c78)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:01.497718 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fad58a-fa39-4dc0-b856-6b3ad8249f9d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:01.549913 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-73fad58a-fa39-4dc0-b856-6b3ad8249f9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2af4d4a69e9d4755bff61401f4349c78'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:01.550630 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-73fad58a-fa39-4dc0-b856-6b3ad8249f9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2af4d4a69e9d4755bff61401f4349c78'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:01.565074 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-73fad58a-fa39-4dc0-b856-6b3ad8249f9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2af4d4a69e9d4755bff61401f4349c78'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:01.565799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-73fad58a-fa39-4dc0-b856-6b3ad8249f9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2af4d4a69e9d4755bff61401f4349c78'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:01.575388 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-73fad58a-fa39-4dc0-b856-6b3ad8249f9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2af4d4a69e9d4755bff61401f4349c78'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:01.594134 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1377/2741] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:43:01 2022] DELETE /identity/v3/users/2af4d4a69e9d4755bff61401f4349c78 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:43:01.781265 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334f267-0805-45d2-85b1-07d090f20360 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:01.795110 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334f267-0805-45d2-85b1-07d090f20360 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:01.857232 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be10b07-e86d-418b-95ef-d9e515104c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:01.858053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be10b07-e86d-418b-95ef-d9e515104c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:01.858337 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be10b07-e86d-418b-95ef-d9e515104c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:01.858659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be10b07-e86d-418b-95ef-d9e515104c30 admin admin] PATH_INFO: `/v3/projects/f18ffa75c23e4276a8b5b4fc3821fb87` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:01.859834 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be10b07-e86d-418b-95ef-d9e515104c30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f18ffa75c23e4276a8b5b4fc3821fb87)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:01.861783 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be10b07-e86d-418b-95ef-d9e515104c30 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:01.909082 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2be10b07-e86d-418b-95ef-d9e515104c30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f18ffa75c23e4276a8b5b4fc3821fb87', 'request_id': 'req-2be10b07-e86d-418b-95ef-d9e515104c30'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:01.915474 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2be10b07-e86d-418b-95ef-d9e515104c30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f18ffa75c23e4276a8b5b4fc3821fb87', 'request_id': 'req-2be10b07-e86d-418b-95ef-d9e515104c30'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:01.920882 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2be10b07-e86d-418b-95ef-d9e515104c30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f18ffa75c23e4276a8b5b4fc3821fb87', 'request_id': 'req-2be10b07-e86d-418b-95ef-d9e515104c30'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:01.933216 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2be10b07-e86d-418b-95ef-d9e515104c30 admin admin] The token cache is being invalidate because project f18ffa75c23e4276a8b5b4fc3821fb87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:43:01.933506 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2be10b07-e86d-418b-95ef-d9e515104c30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:01.934153 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2be10b07-e86d-418b-95ef-d9e515104c30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:01.935734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1365/2742] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:43:01 2022] DELETE /identity/v3/projects/f18ffa75c23e4276a8b5b4fc3821fb87 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:43:01.940827 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bdb51a-938e-4400-ac24-8b9c24d6f66c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:01.941120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bdb51a-938e-4400-ac24-8b9c24d6f66c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:01.941400 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bdb51a-938e-4400-ac24-8b9c24d6f66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:01.959685 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-56bdb51a-938e-4400-ac24-8b9c24d6f66c None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:43:01.961580 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-56bdb51a-938e-4400-ac24-8b9c24d6f66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:43:02.023884 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1378/2743] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:43:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:43:02.031721 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6edb4-36a3-4476-8969-a40dbe8b31c0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:02.032012 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6edb4-36a3-4476-8969-a40dbe8b31c0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:02.032293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f6edb4-36a3-4476-8969-a40dbe8b31c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:02.050152 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-05f6edb4-36a3-4476-8969-a40dbe8b31c0 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:43:02.051381 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-05f6edb4-36a3-4476-8969-a40dbe8b31c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:43:02.059973 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1366/2744] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:43:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:43:02.064219 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bdb51a-938e-4400-ac24-8b9c24d6f66c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:02.074961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bdb51a-938e-4400-ac24-8b9c24d6f66c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:02.079541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffb331-a535-41fa-af33-c5a45e7d0708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:02.080192 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffb331-a535-41fa-af33-c5a45e7d0708 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:02.080479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffb331-a535-41fa-af33-c5a45e7d0708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:02.080759 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffb331-a535-41fa-af33-c5a45e7d0708 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:02.081486 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffb331-a535-41fa-af33-c5a45e7d0708 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:43:02.081794 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffb331-a535-41fa-af33-c5a45e7d0708 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:02.083415 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffb331-a535-41fa-af33-c5a45e7d0708 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:02.095471 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1379/2745] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:43:02.099535 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6edb4-36a3-4476-8969-a40dbe8b31c0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:02.108897 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f6edb4-36a3-4476-8969-a40dbe8b31c0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:02.113536 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac97be2-c69e-4600-aaef-63b12b03cfaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:02.114186 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac97be2-c69e-4600-aaef-63b12b03cfaf admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:02.114504 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac97be2-c69e-4600-aaef-63b12b03cfaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:02.114787 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac97be2-c69e-4600-aaef-63b12b03cfaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:02.115491 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac97be2-c69e-4600-aaef-63b12b03cfaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:02.118479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac97be2-c69e-4600-aaef-63b12b03cfaf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:02.145953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1367/2746] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:43:02 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:43:02.149693 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffb331-a535-41fa-af33-c5a45e7d0708 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:02.159872 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffb331-a535-41fa-af33-c5a45e7d0708 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:02.216274 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b39b7-06bf-4ccc-9025-8471474185b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:02.216965 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b39b7-06bf-4ccc-9025-8471474185b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:02.217282 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b39b7-06bf-4ccc-9025-8471474185b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:02.217609 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b39b7-06bf-4ccc-9025-8471474185b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:02.218274 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72b39b7-06bf-4ccc-9025-8471474185b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:02.220221 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72b39b7-06bf-4ccc-9025-8471474185b8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:02.226871 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:43:02.241257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1380/2747] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:43:02 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:43:02.245850 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac97be2-c69e-4600-aaef-63b12b03cfaf admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:02.256250 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac97be2-c69e-4600-aaef-63b12b03cfaf admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:02.261759 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f55c6-9a87-4869-be41-9ec722f99d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:02.262514 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f55c6-9a87-4869-be41-9ec722f99d3d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:02.262801 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f55c6-9a87-4869-be41-9ec722f99d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:02.263100 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03f55c6-9a87-4869-be41-9ec722f99d3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:02.263828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f55c6-9a87-4869-be41-9ec722f99d3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:02.265418 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03f55c6-9a87-4869-be41-9ec722f99d3d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:02.276371 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1368/2748] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:43:02.280334 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b39b7-06bf-4ccc-9025-8471474185b8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:02.290717 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b39b7-06bf-4ccc-9025-8471474185b8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:02.295301 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170feab6-0be0-4163-a973-fb4f64df4840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:02.295953 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170feab6-0be0-4163-a973-fb4f64df4840 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:02.296239 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170feab6-0be0-4163-a973-fb4f64df4840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:02.296515 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170feab6-0be0-4163-a973-fb4f64df4840 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:02.297240 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170feab6-0be0-4163-a973-fb4f64df4840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:02.298858 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170feab6-0be0-4163-a973-fb4f64df4840 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:02.311348 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1381/2749] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:43:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:43:02.315278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f55c6-9a87-4869-be41-9ec722f99d3d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:02.325626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03f55c6-9a87-4869-be41-9ec722f99d3d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:02.330218 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46be18-f85b-4173-8d0c-01a68a68f898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:02.331185 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46be18-f85b-4173-8d0c-01a68a68f898 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:02.331487 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46be18-f85b-4173-8d0c-01a68a68f898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:02.331744 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46be18-f85b-4173-8d0c-01a68a68f898 admin admin] PATH_INFO: `/v3/projects/4fb6837bd2d4431fb6f9afccd5982175/users/eb34fd2953d74f178e0eb8aa9909f7aa/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:02.346098 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46be18-f85b-4173-8d0c-01a68a68f898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fb6837bd2d4431fb6f9afccd5982175, user_id=eb34fd2953d74f178e0eb8aa9909f7aa, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:02.348813 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46be18-f85b-4173-8d0c-01a68a68f898 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:02.362299 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1369/2750] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:43:02 2022] PUT /identity/v3/projects/4fb6837bd2d4431fb6f9afccd5982175/users/eb34fd2953d74f178e0eb8aa9909f7aa/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:43:02.373634 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a66e6-193e-470d-b98e-59f68ee59973 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:02.373956 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a66e6-193e-470d-b98e-59f68ee59973 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:02.374243 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a66e6-193e-470d-b98e-59f68ee59973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:02.400767 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bc5a66e6-193e-470d-b98e-59f68ee59973 None None] MFA Rules not processed for user `eb34fd2953d74f178e0eb8aa9909f7aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:43:02.401967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bc5a66e6-193e-470d-b98e-59f68ee59973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:43:02.466270 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1382/2751] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:43:02 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:43:02.479224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46be18-f85b-4173-8d0c-01a68a68f898 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:02.480903 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9a46be18-f85b-4173-8d0c-01a68a68f898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:43:02.511978 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:43:02.541438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46be18-f85b-4173-8d0c-01a68a68f898 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:02.545084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df988b54-9fc0-4104-88f5-d33c7cb48722 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:02.545724 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df988b54-9fc0-4104-88f5-d33c7cb48722 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:02.546004 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df988b54-9fc0-4104-88f5-d33c7cb48722 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:02.546282 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df988b54-9fc0-4104-88f5-d33c7cb48722 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:02.555125 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df988b54-9fc0-4104-88f5-d33c7cb48722 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:02.556551 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df988b54-9fc0-4104-88f5-d33c7cb48722 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:02.565085 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1370/2752] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:43:02 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:43:06.095570 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a08692-a9e7-4312-822a-5923453733b3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:06.096061 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a08692-a9e7-4312-822a-5923453733b3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:06.096160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a08692-a9e7-4312-822a-5923453733b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:06.114109 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-99a08692-a9e7-4312-822a-5923453733b3 None None] MFA Rules not processed for user `eb9bae5e3522437182d2650c3fcfdef6`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:43:06.115343 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-99a08692-a9e7-4312-822a-5923453733b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:43:06.169807 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1383/2753] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:43:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:43:11.979511 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df988b54-9fc0-4104-88f5-d33c7cb48722 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:12.041496 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df988b54-9fc0-4104-88f5-d33c7cb48722 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:12.045878 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c448735-7c63-4678-b3c2-015e01b0844a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:12.046763 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c448735-7c63-4678-b3c2-015e01b0844a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:12.047046 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c448735-7c63-4678-b3c2-015e01b0844a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:12.047321 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c448735-7c63-4678-b3c2-015e01b0844a admin admin] PATH_INFO: `/v3/users/eb34fd2953d74f178e0eb8aa9909f7aa` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:12.048485 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c448735-7c63-4678-b3c2-015e01b0844a admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb34fd2953d74f178e0eb8aa9909f7aa)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:12.050401 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c448735-7c63-4678-b3c2-015e01b0844a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:12.096771 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c448735-7c63-4678-b3c2-015e01b0844a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb34fd2953d74f178e0eb8aa9909f7aa'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:12.105147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c448735-7c63-4678-b3c2-015e01b0844a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb34fd2953d74f178e0eb8aa9909f7aa'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:12.117466 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c448735-7c63-4678-b3c2-015e01b0844a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb34fd2953d74f178e0eb8aa9909f7aa'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:12.118111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c448735-7c63-4678-b3c2-015e01b0844a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb34fd2953d74f178e0eb8aa9909f7aa'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:12.118748 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c448735-7c63-4678-b3c2-015e01b0844a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb34fd2953d74f178e0eb8aa9909f7aa'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:12.130711 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1371/2754] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:43:11 2022] DELETE /identity/v3/users/eb34fd2953d74f178e0eb8aa9909f7aa => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:43:12.135670 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5df58-37d7-4779-a2c3-1839c901de86 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:12.136132 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5df58-37d7-4779-a2c3-1839c901de86 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:12.136272 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5df58-37d7-4779-a2c3-1839c901de86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:12.154608 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-90d5df58-37d7-4779-a2c3-1839c901de86 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:43:12.155841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-90d5df58-37d7-4779-a2c3-1839c901de86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:43:12.214338 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1384/2755] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:43:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:43:12.224072 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c448735-7c63-4678-b3c2-015e01b0844a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:12.225480 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6c448735-7c63-4678-b3c2-015e01b0844a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:43:12.282283 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:43:12.286097 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c448735-7c63-4678-b3c2-015e01b0844a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:12.291097 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1780c7a8-cc3e-4cd9-9e4a-db6029bb01c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:12.291828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780c7a8-cc3e-4cd9-9e4a-db6029bb01c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:12.292461 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780c7a8-cc3e-4cd9-9e4a-db6029bb01c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:12.292911 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1780c7a8-cc3e-4cd9-9e4a-db6029bb01c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:12.302583 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1780c7a8-cc3e-4cd9-9e4a-db6029bb01c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:12.304016 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1780c7a8-cc3e-4cd9-9e4a-db6029bb01c9 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:12.313528 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1372/2756] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:43:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:43:12.594084 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d5df58-37d7-4779-a2c3-1839c901de86 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:12.595568 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-90d5df58-37d7-4779-a2c3-1839c901de86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:43:12.608372 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d5df58-37d7-4779-a2c3-1839c901de86 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:12.612961 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee66be-7fda-4aef-bae5-7484aba00047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:12.613830 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee66be-7fda-4aef-bae5-7484aba00047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:12.614118 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee66be-7fda-4aef-bae5-7484aba00047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:12.614396 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee66be-7fda-4aef-bae5-7484aba00047 admin admin] PATH_INFO: `/v3/projects/4fb6837bd2d4431fb6f9afccd5982175` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:12.615726 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee66be-7fda-4aef-bae5-7484aba00047 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fb6837bd2d4431fb6f9afccd5982175)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:12.618632 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee66be-7fda-4aef-bae5-7484aba00047 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:12.627365 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:43:12.668910 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-82ee66be-7fda-4aef-bae5-7484aba00047 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fb6837bd2d4431fb6f9afccd5982175', 'request_id': 'req-82ee66be-7fda-4aef-bae5-7484aba00047'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:12.675902 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-82ee66be-7fda-4aef-bae5-7484aba00047 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fb6837bd2d4431fb6f9afccd5982175', 'request_id': 'req-82ee66be-7fda-4aef-bae5-7484aba00047'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:12.683462 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-82ee66be-7fda-4aef-bae5-7484aba00047 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fb6837bd2d4431fb6f9afccd5982175', 'request_id': 'req-82ee66be-7fda-4aef-bae5-7484aba00047'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:12.694108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-82ee66be-7fda-4aef-bae5-7484aba00047 admin admin] The token cache is being invalidate because project 4fb6837bd2d4431fb6f9afccd5982175 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:43:12.694473 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-82ee66be-7fda-4aef-bae5-7484aba00047 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:12.696294 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-82ee66be-7fda-4aef-bae5-7484aba00047 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:12.701123 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1385/2757] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:43:12 2022] DELETE /identity/v3/projects/4fb6837bd2d4431fb6f9afccd5982175 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:43:12.702671 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e1ca2-7b52-49df-a352-6b1bd7ecaa72 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:12.702958 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e1ca2-7b52-49df-a352-6b1bd7ecaa72 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:12.703239 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2e1ca2-7b52-49df-a352-6b1bd7ecaa72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:12.721263 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bd2e1ca2-7b52-49df-a352-6b1bd7ecaa72 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:43:12.722226 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bd2e1ca2-7b52-49df-a352-6b1bd7ecaa72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:43:12.803233 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1373/2758] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:43:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:43:12.808287 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937b511-dfae-4af0-8036-bbf021e3a694 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:12.808588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937b511-dfae-4af0-8036-bbf021e3a694 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:12.808870 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8937b511-dfae-4af0-8036-bbf021e3a694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:12.829429 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8937b511-dfae-4af0-8036-bbf021e3a694 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:43:12.830963 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8937b511-dfae-4af0-8036-bbf021e3a694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:43:12.839439 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1386/2759] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:43:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:43:12.843718 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e1ca2-7b52-49df-a352-6b1bd7ecaa72 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:12.855711 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2e1ca2-7b52-49df-a352-6b1bd7ecaa72 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:12.860639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2ebb0-54c6-474d-a859-e3d2884bbf14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:12.861382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2ebb0-54c6-474d-a859-e3d2884bbf14 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:12.861737 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2ebb0-54c6-474d-a859-e3d2884bbf14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:12.862105 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df2ebb0-54c6-474d-a859-e3d2884bbf14 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:12.863316 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df2ebb0-54c6-474d-a859-e3d2884bbf14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:43:12.863620 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df2ebb0-54c6-474d-a859-e3d2884bbf14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:12.865434 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df2ebb0-54c6-474d-a859-e3d2884bbf14 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:12.879283 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1374/2760] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 29 12:43:12.883598 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8937b511-dfae-4af0-8036-bbf021e3a694 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:12.893780 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8937b511-dfae-4af0-8036-bbf021e3a694 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:12.899581 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cecf40d-f7de-4e21-86b7-1d3e936918e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:12.900326 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cecf40d-f7de-4e21-86b7-1d3e936918e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:12.900618 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cecf40d-f7de-4e21-86b7-1d3e936918e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:12.900897 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cecf40d-f7de-4e21-86b7-1d3e936918e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:12.901647 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cecf40d-f7de-4e21-86b7-1d3e936918e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:12.903602 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cecf40d-f7de-4e21-86b7-1d3e936918e2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:12.931360 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1387/2761] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:43:12 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:43:12.935719 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2ebb0-54c6-474d-a859-e3d2884bbf14 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:12.945549 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df2ebb0-54c6-474d-a859-e3d2884bbf14 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:13.001980 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2bc8f-ebea-41fe-a65e-31d2e59c9bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:13.002688 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2bc8f-ebea-41fe-a65e-31d2e59c9bef admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:13.002975 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2bc8f-ebea-41fe-a65e-31d2e59c9bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:13.003258 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2bc8f-ebea-41fe-a65e-31d2e59c9bef admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:13.003949 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f2bc8f-ebea-41fe-a65e-31d2e59c9bef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:13.005872 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f2bc8f-ebea-41fe-a65e-31d2e59c9bef admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:13.026398 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1375/2762] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:43:12 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:43:13.030442 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cecf40d-f7de-4e21-86b7-1d3e936918e2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:13.041822 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cecf40d-f7de-4e21-86b7-1d3e936918e2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:13.046386 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59cbd5-9d7b-44ce-9477-608585ebb4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:13.047069 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59cbd5-9d7b-44ce-9477-608585ebb4e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:13.047356 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59cbd5-9d7b-44ce-9477-608585ebb4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:13.047642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d59cbd5-9d7b-44ce-9477-608585ebb4e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:13.048362 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d59cbd5-9d7b-44ce-9477-608585ebb4e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:13.049948 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d59cbd5-9d7b-44ce-9477-608585ebb4e3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:13.061393 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1388/2763] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:43:13.065248 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2bc8f-ebea-41fe-a65e-31d2e59c9bef admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:13.075352 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2bc8f-ebea-41fe-a65e-31d2e59c9bef admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:13.079909 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b220f3-4e4a-4446-aa2d-179ab2fe9e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:13.080554 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b220f3-4e4a-4446-aa2d-179ab2fe9e33 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:13.080837 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b220f3-4e4a-4446-aa2d-179ab2fe9e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:13.081133 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b220f3-4e4a-4446-aa2d-179ab2fe9e33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:13.081829 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b220f3-4e4a-4446-aa2d-179ab2fe9e33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:13.083516 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b220f3-4e4a-4446-aa2d-179ab2fe9e33 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:13.093952 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1376/2764] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:43:13.098131 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59cbd5-9d7b-44ce-9477-608585ebb4e3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:13.108940 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d59cbd5-9d7b-44ce-9477-608585ebb4e3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:13.113609 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a217e5-c017-44fd-9f6e-0d9d7fffca65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:13.114525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a217e5-c017-44fd-9f6e-0d9d7fffca65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:13.114813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a217e5-c017-44fd-9f6e-0d9d7fffca65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:13.115093 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a217e5-c017-44fd-9f6e-0d9d7fffca65 admin admin] PATH_INFO: `/v3/projects/de018c75e8bc478f8c302b152fb899f7/users/60db93f70cd64787b97ff1015c9ec9cf/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:13.131033 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a217e5-c017-44fd-9f6e-0d9d7fffca65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de018c75e8bc478f8c302b152fb899f7, user_id=60db93f70cd64787b97ff1015c9ec9cf, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:13.134290 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a217e5-c017-44fd-9f6e-0d9d7fffca65 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:13.147695 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1389/2765] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:43:13 2022] PUT /identity/v3/projects/de018c75e8bc478f8c302b152fb899f7/users/60db93f70cd64787b97ff1015c9ec9cf/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 29 12:43:13.158314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad31ed-7ab5-4890-b664-b20b6bd77008 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:13.158638 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad31ed-7ab5-4890-b664-b20b6bd77008 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:13.158920 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad31ed-7ab5-4890-b664-b20b6bd77008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:13.187052 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5fad31ed-7ab5-4890-b664-b20b6bd77008 None None] MFA Rules not processed for user `60db93f70cd64787b97ff1015c9ec9cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:43:13.188248 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5fad31ed-7ab5-4890-b664-b20b6bd77008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:43:13.234374 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:43:13.255028 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1377/2766] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:43:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:43:13.259668 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a217e5-c017-44fd-9f6e-0d9d7fffca65 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:13.323708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a217e5-c017-44fd-9f6e-0d9d7fffca65 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:13.328264 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f34f7-7f12-4ea1-b98c-4fb1f497f087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:13.328918 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f34f7-7f12-4ea1-b98c-4fb1f497f087 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:13.329201 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f34f7-7f12-4ea1-b98c-4fb1f497f087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:13.329481 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9f34f7-7f12-4ea1-b98c-4fb1f497f087 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:13.330175 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f34f7-7f12-4ea1-b98c-4fb1f497f087 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:13.332094 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9f34f7-7f12-4ea1-b98c-4fb1f497f087 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:13.361301 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1390/2767] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:43:13 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:43:13.362871 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fad31ed-7ab5-4890-b664-b20b6bd77008 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:13.373929 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fad31ed-7ab5-4890-b664-b20b6bd77008 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:13.430916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55906ca-a2bc-4ba6-b908-2f2f86486e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:13.431579 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55906ca-a2bc-4ba6-b908-2f2f86486e52 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:13.431865 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55906ca-a2bc-4ba6-b908-2f2f86486e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:13.432145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55906ca-a2bc-4ba6-b908-2f2f86486e52 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:13.432826 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55906ca-a2bc-4ba6-b908-2f2f86486e52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:13.434760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55906ca-a2bc-4ba6-b908-2f2f86486e52 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:13.454158 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1378/2768] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:43:13 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:43:13.458137 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f34f7-7f12-4ea1-b98c-4fb1f497f087 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:13.469310 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9f34f7-7f12-4ea1-b98c-4fb1f497f087 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:13.474259 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7e3086-551a-4873-a7b4-55211c414568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:13.475100 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e3086-551a-4873-a7b4-55211c414568 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:13.475397 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e3086-551a-4873-a7b4-55211c414568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:13.476338 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e3086-551a-4873-a7b4-55211c414568 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:13.477085 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7e3086-551a-4873-a7b4-55211c414568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:13.478713 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7e3086-551a-4873-a7b4-55211c414568 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:13.490970 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1391/2769] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:43:13.494982 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55906ca-a2bc-4ba6-b908-2f2f86486e52 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:13.504881 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55906ca-a2bc-4ba6-b908-2f2f86486e52 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:13.509637 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6691ed2-0b18-4c2f-adb6-3b912b543ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:13.510485 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6691ed2-0b18-4c2f-adb6-3b912b543ca1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:13.510772 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6691ed2-0b18-4c2f-adb6-3b912b543ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:13.511048 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6691ed2-0b18-4c2f-adb6-3b912b543ca1 admin admin] PATH_INFO: `/v3/projects/a0a70c5f00b245f29edf24257a29a022/users/e227fa0f5b714ca4b33f501c2a827a61/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:13.526150 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6691ed2-0b18-4c2f-adb6-3b912b543ca1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0a70c5f00b245f29edf24257a29a022, user_id=e227fa0f5b714ca4b33f501c2a827a61, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:13.528820 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6691ed2-0b18-4c2f-adb6-3b912b543ca1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:13.541148 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1379/2770] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:43:13 2022] PUT /identity/v3/projects/a0a70c5f00b245f29edf24257a29a022/users/e227fa0f5b714ca4b33f501c2a827a61/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:43:13.545031 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7e3086-551a-4873-a7b4-55211c414568 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:13.556329 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7e3086-551a-4873-a7b4-55211c414568 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:13.559889 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:43:13.614896 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ebd57-a9a0-4c8c-85b2-6a65b89a341b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:13.615561 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ebd57-a9a0-4c8c-85b2-6a65b89a341b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:13.615848 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ebd57-a9a0-4c8c-85b2-6a65b89a341b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:13.616128 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ebd57-a9a0-4c8c-85b2-6a65b89a341b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:13.618010 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ebd57-a9a0-4c8c-85b2-6a65b89a341b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:13.619761 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ebd57-a9a0-4c8c-85b2-6a65b89a341b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:13.629450 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1392/2771] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:43:13.633600 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6691ed2-0b18-4c2f-adb6-3b912b543ca1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:13.643887 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6691ed2-0b18-4c2f-adb6-3b912b543ca1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:13.648819 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520bce66-0267-469a-981c-eab3cbc66ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:13.649694 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bce66-0267-469a-981c-eab3cbc66ee6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:13.649979 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bce66-0267-469a-981c-eab3cbc66ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:13.650256 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bce66-0267-469a-981c-eab3cbc66ee6 admin admin] PATH_INFO: `/v3/projects/a0a70c5f00b245f29edf24257a29a022/users/e227fa0f5b714ca4b33f501c2a827a61/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:13.652233 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520bce66-0267-469a-981c-eab3cbc66ee6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0a70c5f00b245f29edf24257a29a022, user_id=e227fa0f5b714ca4b33f501c2a827a61, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:13.654927 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520bce66-0267-469a-981c-eab3cbc66ee6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:13.665455 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1380/2772] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:43:13 2022] PUT /identity/v3/projects/a0a70c5f00b245f29edf24257a29a022/users/e227fa0f5b714ca4b33f501c2a827a61/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:43:13.676552 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a364fc-650a-40ef-95c6-aacaa351e038 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:13.676843 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a364fc-650a-40ef-95c6-aacaa351e038 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:13.677120 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a364fc-650a-40ef-95c6-aacaa351e038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:13.703761 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-17a364fc-650a-40ef-95c6-aacaa351e038 None None] MFA Rules not processed for user `e227fa0f5b714ca4b33f501c2a827a61`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:43:13.704951 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-17a364fc-650a-40ef-95c6-aacaa351e038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:43:13.771075 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1393/2773] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:43:13 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:43:13.782517 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520bce66-0267-469a-981c-eab3cbc66ee6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:13.784211 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-520bce66-0267-469a-981c-eab3cbc66ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:43:13.842199 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520bce66-0267-469a-981c-eab3cbc66ee6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:13.845824 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40c3a1-beba-43ce-829e-263ff936a329 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:13.846482 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40c3a1-beba-43ce-829e-263ff936a329 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:13.846771 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40c3a1-beba-43ce-829e-263ff936a329 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:13.847054 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40c3a1-beba-43ce-829e-263ff936a329 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:13.856907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40c3a1-beba-43ce-829e-263ff936a329 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:13.858329 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40c3a1-beba-43ce-829e-263ff936a329 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:13.868611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1381/2774] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:43:13 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:43:13.938887 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a364fc-650a-40ef-95c6-aacaa351e038 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:13.948786 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a364fc-650a-40ef-95c6-aacaa351e038 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:13.952715 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a02c9-84bb-41bf-a18e-f47fd6ecf98e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:13.953375 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a02c9-84bb-41bf-a18e-f47fd6ecf98e service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:13.953660 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a02c9-84bb-41bf-a18e-f47fd6ecf98e service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:13.953939 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a02c9-84bb-41bf-a18e-f47fd6ecf98e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:14.013806 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a02c9-84bb-41bf-a18e-f47fd6ecf98e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:14.015397 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a02c9-84bb-41bf-a18e-f47fd6ecf98e service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:14.025547 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1394/2775] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:43:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:43:23.863451 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=76,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:43:24.020359 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:43:36.076047 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40c3a1-beba-43ce-829e-263ff936a329 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:36.077509 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ac40c3a1-beba-43ce-829e-263ff936a329 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:43:36.089701 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40c3a1-beba-43ce-829e-263ff936a329 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:36.093575 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c00b1-9c01-4e54-bd35-23a6308721de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:36.094253 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c00b1-9c01-4e54-bd35-23a6308721de service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:36.094581 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c00b1-9c01-4e54-bd35-23a6308721de service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:36.094890 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c00b1-9c01-4e54-bd35-23a6308721de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:36.096803 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-931c00b1-9c01-4e54-bd35-23a6308721de service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:43:36.172775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c00b1-9c01-4e54-bd35-23a6308721de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:36.174224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c00b1-9c01-4e54-bd35-23a6308721de service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:36.185814 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1382/2776] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:43:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:43:38.780045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a02c9-84bb-41bf-a18e-f47fd6ecf98e service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:38.781749 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-839a02c9-84bb-41bf-a18e-f47fd6ecf98e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:43:38.841457 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a02c9-84bb-41bf-a18e-f47fd6ecf98e service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:38.846248 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f1fc0a-ded2-4482-b61f-8a08784669d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:38.846248 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1fc0a-ded2-4482-b61f-8a08784669d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:38.847967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1fc0a-ded2-4482-b61f-8a08784669d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:38.847967 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f1fc0a-ded2-4482-b61f-8a08784669d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:38.849162 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-69f1fc0a-ded2-4482-b61f-8a08784669d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:43:38.863631 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f1fc0a-ded2-4482-b61f-8a08784669d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:38.865059 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f1fc0a-ded2-4482-b61f-8a08784669d1 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:38.876660 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1395/2777] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:43:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:43:46.181191 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=34 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:43:48.871756 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=26 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:43:56.480734 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c00b1-9c01-4e54-bd35-23a6308721de service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:56.542650 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c00b1-9c01-4e54-bd35-23a6308721de service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:56.547214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874872dc-e147-426b-9d25-8db00e2f255d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:56.548076 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874872dc-e147-426b-9d25-8db00e2f255d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:56.548364 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874872dc-e147-426b-9d25-8db00e2f255d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:56.548654 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874872dc-e147-426b-9d25-8db00e2f255d admin admin] PATH_INFO: `/v3/users/60db93f70cd64787b97ff1015c9ec9cf` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:56.549832 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874872dc-e147-426b-9d25-8db00e2f255d admin admin] RBAC: Authorizing `identity:delete_user(user_id=60db93f70cd64787b97ff1015c9ec9cf)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:56.551795 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874872dc-e147-426b-9d25-8db00e2f255d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:56.597216 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-874872dc-e147-426b-9d25-8db00e2f255d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60db93f70cd64787b97ff1015c9ec9cf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:56.604530 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-874872dc-e147-426b-9d25-8db00e2f255d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60db93f70cd64787b97ff1015c9ec9cf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:56.616468 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-874872dc-e147-426b-9d25-8db00e2f255d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60db93f70cd64787b97ff1015c9ec9cf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:56.617103 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-874872dc-e147-426b-9d25-8db00e2f255d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60db93f70cd64787b97ff1015c9ec9cf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:56.617697 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-874872dc-e147-426b-9d25-8db00e2f255d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60db93f70cd64787b97ff1015c9ec9cf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:56.630104 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1383/2778] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:43:56 2022] DELETE /identity/v3/users/60db93f70cd64787b97ff1015c9ec9cf => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:43:56.634600 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15751962-4d3e-4377-9003-6177af96ee2c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:56.635078 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15751962-4d3e-4377-9003-6177af96ee2c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:56.635177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15751962-4d3e-4377-9003-6177af96ee2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:56.652735 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-15751962-4d3e-4377-9003-6177af96ee2c None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:43:56.653920 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-15751962-4d3e-4377-9003-6177af96ee2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:43:56.711369 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1396/2779] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:43:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:43:56.720821 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874872dc-e147-426b-9d25-8db00e2f255d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:56.722508 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-874872dc-e147-426b-9d25-8db00e2f255d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:43:56.780539 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874872dc-e147-426b-9d25-8db00e2f255d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:56.784170 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9203380-a3c7-4a60-acb8-bbff2fbfa69a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:56.784802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9203380-a3c7-4a60-acb8-bbff2fbfa69a service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:56.785084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9203380-a3c7-4a60-acb8-bbff2fbfa69a service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:56.785360 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9203380-a3c7-4a60-acb8-bbff2fbfa69a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:56.794498 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9203380-a3c7-4a60-acb8-bbff2fbfa69a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:56.795874 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9203380-a3c7-4a60-acb8-bbff2fbfa69a service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:56.805445 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1384/2780] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:43:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:43:57.077664 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15751962-4d3e-4377-9003-6177af96ee2c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:57.079273 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-15751962-4d3e-4377-9003-6177af96ee2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:43:57.091467 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15751962-4d3e-4377-9003-6177af96ee2c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:57.096098 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f639d2-3fb0-40d4-89ad-464b9ba9f741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:57.096975 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f639d2-3fb0-40d4-89ad-464b9ba9f741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:57.097262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f639d2-3fb0-40d4-89ad-464b9ba9f741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:57.097541 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f639d2-3fb0-40d4-89ad-464b9ba9f741 admin admin] PATH_INFO: `/v3/projects/de018c75e8bc478f8c302b152fb899f7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:57.098769 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f639d2-3fb0-40d4-89ad-464b9ba9f741 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de018c75e8bc478f8c302b152fb899f7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:57.100686 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f639d2-3fb0-40d4-89ad-464b9ba9f741 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:57.148823 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-91f639d2-3fb0-40d4-89ad-464b9ba9f741 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de018c75e8bc478f8c302b152fb899f7', 'request_id': 'req-91f639d2-3fb0-40d4-89ad-464b9ba9f741'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:57.156739 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-91f639d2-3fb0-40d4-89ad-464b9ba9f741 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de018c75e8bc478f8c302b152fb899f7', 'request_id': 'req-91f639d2-3fb0-40d4-89ad-464b9ba9f741'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:57.163458 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-91f639d2-3fb0-40d4-89ad-464b9ba9f741 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de018c75e8bc478f8c302b152fb899f7', 'request_id': 'req-91f639d2-3fb0-40d4-89ad-464b9ba9f741'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:57.174682 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-91f639d2-3fb0-40d4-89ad-464b9ba9f741 admin admin] The token cache is being invalidate because project de018c75e8bc478f8c302b152fb899f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:43:57.174970 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-91f639d2-3fb0-40d4-89ad-464b9ba9f741 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:57.175599 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-91f639d2-3fb0-40d4-89ad-464b9ba9f741 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:57.177147 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1397/2781] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:43:57 2022] DELETE /identity/v3/projects/de018c75e8bc478f8c302b152fb899f7 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:43:57.181210 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9203380-a3c7-4a60-acb8-bbff2fbfa69a service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:57.192491 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9203380-a3c7-4a60-acb8-bbff2fbfa69a service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:57.210036 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:43:57.253547 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f392e6fe-2ccf-453d-a04c-cc691df94d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:57.254441 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392e6fe-2ccf-453d-a04c-cc691df94d5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:57.254743 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392e6fe-2ccf-453d-a04c-cc691df94d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:57.255046 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f392e6fe-2ccf-453d-a04c-cc691df94d5a admin admin] PATH_INFO: `/v3/users/e227fa0f5b714ca4b33f501c2a827a61` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:57.256266 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f392e6fe-2ccf-453d-a04c-cc691df94d5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e227fa0f5b714ca4b33f501c2a827a61)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:57.258185 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f392e6fe-2ccf-453d-a04c-cc691df94d5a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:57.304462 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f392e6fe-2ccf-453d-a04c-cc691df94d5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e227fa0f5b714ca4b33f501c2a827a61'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:57.312245 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f392e6fe-2ccf-453d-a04c-cc691df94d5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e227fa0f5b714ca4b33f501c2a827a61'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:57.324608 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f392e6fe-2ccf-453d-a04c-cc691df94d5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e227fa0f5b714ca4b33f501c2a827a61'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:57.325240 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f392e6fe-2ccf-453d-a04c-cc691df94d5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e227fa0f5b714ca4b33f501c2a827a61'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:57.325840 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f392e6fe-2ccf-453d-a04c-cc691df94d5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e227fa0f5b714ca4b33f501c2a827a61'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:57.337656 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1385/2782] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:43:57 2022] DELETE /identity/v3/users/e227fa0f5b714ca4b33f501c2a827a61 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:43:57.608091 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f639d2-3fb0-40d4-89ad-464b9ba9f741 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:57.609500 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-91f639d2-3fb0-40d4-89ad-464b9ba9f741 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:43:57.623007 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f639d2-3fb0-40d4-89ad-464b9ba9f741 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:57.680564 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c935260-5ca4-4b89-94b0-56e522ded3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:57.681438 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c935260-5ca4-4b89-94b0-56e522ded3bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:57.681730 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c935260-5ca4-4b89-94b0-56e522ded3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:57.682153 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c935260-5ca4-4b89-94b0-56e522ded3bf admin admin] PATH_INFO: `/v3/projects/a0a70c5f00b245f29edf24257a29a022` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:57.683706 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c935260-5ca4-4b89-94b0-56e522ded3bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0a70c5f00b245f29edf24257a29a022)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:57.685554 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c935260-5ca4-4b89-94b0-56e522ded3bf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:57.715317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:43:57.733154 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8c935260-5ca4-4b89-94b0-56e522ded3bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0a70c5f00b245f29edf24257a29a022', 'request_id': 'req-8c935260-5ca4-4b89-94b0-56e522ded3bf'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:57.739924 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8c935260-5ca4-4b89-94b0-56e522ded3bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0a70c5f00b245f29edf24257a29a022', 'request_id': 'req-8c935260-5ca4-4b89-94b0-56e522ded3bf'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:57.746059 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8c935260-5ca4-4b89-94b0-56e522ded3bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0a70c5f00b245f29edf24257a29a022', 'request_id': 'req-8c935260-5ca4-4b89-94b0-56e522ded3bf'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:57.756327 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8c935260-5ca4-4b89-94b0-56e522ded3bf admin admin] The token cache is being invalidate because project a0a70c5f00b245f29edf24257a29a022 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:43:57.756613 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8c935260-5ca4-4b89-94b0-56e522ded3bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:57.757233 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8c935260-5ca4-4b89-94b0-56e522ded3bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:43:57.758791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1398/2783] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:43:57 2022] DELETE /identity/v3/projects/a0a70c5f00b245f29edf24257a29a022 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:43:57.764361 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863fd32-6b06-49c8-87c7-d4675bf46039 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:57.764832 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863fd32-6b06-49c8-87c7-d4675bf46039 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:57.765063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863fd32-6b06-49c8-87c7-d4675bf46039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:57.784477 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7863fd32-6b06-49c8-87c7-d4675bf46039 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:43:57.785666 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7863fd32-6b06-49c8-87c7-d4675bf46039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:43:57.844671 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1386/2784] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:43:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:43:57.852354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f1cd9-e7d5-4255-9cf0-a4713bb60d6b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:57.852642 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f1cd9-e7d5-4255-9cf0-a4713bb60d6b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:57.852919 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081f1cd9-e7d5-4255-9cf0-a4713bb60d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:57.871058 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-081f1cd9-e7d5-4255-9cf0-a4713bb60d6b None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:43:57.872248 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-081f1cd9-e7d5-4255-9cf0-a4713bb60d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:43:57.880190 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1399/2785] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:43:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:43:57.884333 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863fd32-6b06-49c8-87c7-d4675bf46039 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:57.894501 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863fd32-6b06-49c8-87c7-d4675bf46039 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:57.899284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6014031-9831-4feb-8979-e1133eb92542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:57.899933 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6014031-9831-4feb-8979-e1133eb92542 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:57.900219 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6014031-9831-4feb-8979-e1133eb92542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:57.900497 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6014031-9831-4feb-8979-e1133eb92542 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:57.901840 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6014031-9831-4feb-8979-e1133eb92542 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:43:57.902146 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6014031-9831-4feb-8979-e1133eb92542 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:57.903789 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6014031-9831-4feb-8979-e1133eb92542 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:57.915136 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1387/2786] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:43:57.919453 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f1cd9-e7d5-4255-9cf0-a4713bb60d6b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:57.928644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081f1cd9-e7d5-4255-9cf0-a4713bb60d6b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:57.933482 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c59ba1-fa67-4be8-8961-0962cd3ad6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:57.934177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c59ba1-fa67-4be8-8961-0962cd3ad6e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:57.934413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c59ba1-fa67-4be8-8961-0962cd3ad6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:57.934732 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c59ba1-fa67-4be8-8961-0962cd3ad6e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:57.935446 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c59ba1-fa67-4be8-8961-0962cd3ad6e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:57.937325 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c59ba1-fa67-4be8-8961-0962cd3ad6e3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:57.962866 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1400/2787] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:43:57 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:43:57.967817 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6014031-9831-4feb-8979-e1133eb92542 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:57.977657 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6014031-9831-4feb-8979-e1133eb92542 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:58.003088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:43:58.035069 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eeb793-08c5-4ae2-9e49-33e362597acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:58.035731 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eeb793-08c5-4ae2-9e49-33e362597acf admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:58.036013 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eeb793-08c5-4ae2-9e49-33e362597acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:58.036294 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eeb793-08c5-4ae2-9e49-33e362597acf admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:58.037011 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eeb793-08c5-4ae2-9e49-33e362597acf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:58.038934 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eeb793-08c5-4ae2-9e49-33e362597acf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:58.059473 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1388/2788] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:43:57 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:43:58.063659 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c59ba1-fa67-4be8-8961-0962cd3ad6e3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:58.074890 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c59ba1-fa67-4be8-8961-0962cd3ad6e3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:58.079590 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e83c67-399a-4e21-8b23-3e242bfeeb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:58.080249 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e83c67-399a-4e21-8b23-3e242bfeeb5b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:58.080535 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e83c67-399a-4e21-8b23-3e242bfeeb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:58.080812 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e83c67-399a-4e21-8b23-3e242bfeeb5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:58.081531 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e83c67-399a-4e21-8b23-3e242bfeeb5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:58.083153 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e83c67-399a-4e21-8b23-3e242bfeeb5b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:58.094051 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1401/2789] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:43:58.097976 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eeb793-08c5-4ae2-9e49-33e362597acf admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:58.108324 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eeb793-08c5-4ae2-9e49-33e362597acf admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:58.112836 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048891f7-9791-418e-997d-644f85869bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:58.113483 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048891f7-9791-418e-997d-644f85869bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:58.113766 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048891f7-9791-418e-997d-644f85869bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:58.114040 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048891f7-9791-418e-997d-644f85869bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:58.114784 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048891f7-9791-418e-997d-644f85869bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:58.116464 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048891f7-9791-418e-997d-644f85869bc6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:58.127004 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1389/2790] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:43:58.131017 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e83c67-399a-4e21-8b23-3e242bfeeb5b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:58.141590 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e83c67-399a-4e21-8b23-3e242bfeeb5b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:58.146346 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcddbf3-1b52-49ec-b436-258a47e8e08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:58.147244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcddbf3-1b52-49ec-b436-258a47e8e08d admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:58.147528 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcddbf3-1b52-49ec-b436-258a47e8e08d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:58.147802 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcddbf3-1b52-49ec-b436-258a47e8e08d admin admin] PATH_INFO: `/v3/projects/b523ac024205409f82db89412748ae1b/users/73f0cada5a124f36b1b0a8eb36bc85c2/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:58.163052 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcddbf3-1b52-49ec-b436-258a47e8e08d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b523ac024205409f82db89412748ae1b, user_id=73f0cada5a124f36b1b0a8eb36bc85c2, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:58.165657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcddbf3-1b52-49ec-b436-258a47e8e08d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:58.178226 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1402/2791] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:43:58 2022] PUT /identity/v3/projects/b523ac024205409f82db89412748ae1b/users/73f0cada5a124f36b1b0a8eb36bc85c2/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:43:58.240227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f159b9-7817-4204-86f9-e1a48bc0f6b1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:58.240448 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f159b9-7817-4204-86f9-e1a48bc0f6b1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:58.240693 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f159b9-7817-4204-86f9-e1a48bc0f6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:58.267680 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-88f159b9-7817-4204-86f9-e1a48bc0f6b1 None None] MFA Rules not processed for user `73f0cada5a124f36b1b0a8eb36bc85c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:43:58.268867 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-88f159b9-7817-4204-86f9-e1a48bc0f6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:43:58.332860 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1390/2792] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:43:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:43:58.337381 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcddbf3-1b52-49ec-b436-258a47e8e08d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:58.399412 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcddbf3-1b52-49ec-b436-258a47e8e08d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:58.404487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4d0b0-a5d2-4f8e-b0d8-897f5600d57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:58.405140 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4d0b0-a5d2-4f8e-b0d8-897f5600d57f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:58.405428 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4d0b0-a5d2-4f8e-b0d8-897f5600d57f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:58.405702 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4d0b0-a5d2-4f8e-b0d8-897f5600d57f admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:58.406397 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4d0b0-a5d2-4f8e-b0d8-897f5600d57f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:58.408355 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4d0b0-a5d2-4f8e-b0d8-897f5600d57f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:58.433523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1403/2793] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:43:58 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:43:58.437595 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f159b9-7817-4204-86f9-e1a48bc0f6b1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:58.449579 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f159b9-7817-4204-86f9-e1a48bc0f6b1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:58.506113 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9211b-fc5b-4ef6-ad17-734d318dacec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:58.506825 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9211b-fc5b-4ef6-ad17-734d318dacec admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:58.507112 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9211b-fc5b-4ef6-ad17-734d318dacec admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:58.507387 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9211b-fc5b-4ef6-ad17-734d318dacec admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:58.508083 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9211b-fc5b-4ef6-ad17-734d318dacec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:58.509997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b9211b-fc5b-4ef6-ad17-734d318dacec admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:58.529183 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1391/2794] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:43:58 2022] POST /identity/v3/users => generated 298 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:43:58.533134 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4d0b0-a5d2-4f8e-b0d8-897f5600d57f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:58.543789 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4d0b0-a5d2-4f8e-b0d8-897f5600d57f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:58.548358 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b34e15-6dde-4a7c-a810-7adafb2ab1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:58.549012 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b34e15-6dde-4a7c-a810-7adafb2ab1a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:58.549300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b34e15-6dde-4a7c-a810-7adafb2ab1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:58.549578 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b34e15-6dde-4a7c-a810-7adafb2ab1a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:58.550300 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b34e15-6dde-4a7c-a810-7adafb2ab1a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:58.551930 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b34e15-6dde-4a7c-a810-7adafb2ab1a9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:58.562970 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1404/2795] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:43:58.566904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9211b-fc5b-4ef6-ad17-734d318dacec admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:58.576664 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9211b-fc5b-4ef6-ad17-734d318dacec admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:58.581174 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79ac1d-da5e-489a-9d0b-35de49be8c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:58.582040 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79ac1d-da5e-489a-9d0b-35de49be8c7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:58.582320 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79ac1d-da5e-489a-9d0b-35de49be8c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:58.582627 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79ac1d-da5e-489a-9d0b-35de49be8c7c admin admin] PATH_INFO: `/v3/projects/8c1583b00d9d4cb79051c90ca8501244/users/33fa0ec3fffe42bab594f1d8faf0fd76/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:58.596976 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd79ac1d-da5e-489a-9d0b-35de49be8c7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c1583b00d9d4cb79051c90ca8501244, user_id=33fa0ec3fffe42bab594f1d8faf0fd76, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:58.599669 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd79ac1d-da5e-489a-9d0b-35de49be8c7c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:58.612895 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1392/2796] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:43:58 2022] PUT /identity/v3/projects/8c1583b00d9d4cb79051c90ca8501244/users/33fa0ec3fffe42bab594f1d8faf0fd76/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:43:58.617140 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b34e15-6dde-4a7c-a810-7adafb2ab1a9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:58.627635 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b34e15-6dde-4a7c-a810-7adafb2ab1a9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:58.647189 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:43:58.686509 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38651666-cf20-462d-9e64-761965b035df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:58.687177 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38651666-cf20-462d-9e64-761965b035df admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:58.687464 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38651666-cf20-462d-9e64-761965b035df admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:58.687770 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38651666-cf20-462d-9e64-761965b035df admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:58.688493 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38651666-cf20-462d-9e64-761965b035df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:58.690081 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38651666-cf20-462d-9e64-761965b035df admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:58.699691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1405/2797] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:43:58.703983 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79ac1d-da5e-489a-9d0b-35de49be8c7c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:58.709405 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:43:58.715851 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79ac1d-da5e-489a-9d0b-35de49be8c7c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:58.720853 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc0030-ca1b-42b6-b40a-b05e188291ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:58.721775 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc0030-ca1b-42b6-b40a-b05e188291ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:58.722089 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc0030-ca1b-42b6-b40a-b05e188291ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:58.722372 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc0030-ca1b-42b6-b40a-b05e188291ca admin admin] PATH_INFO: `/v3/projects/8c1583b00d9d4cb79051c90ca8501244/users/33fa0ec3fffe42bab594f1d8faf0fd76/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:58.724397 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bc0030-ca1b-42b6-b40a-b05e188291ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c1583b00d9d4cb79051c90ca8501244, user_id=33fa0ec3fffe42bab594f1d8faf0fd76, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:58.727076 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bc0030-ca1b-42b6-b40a-b05e188291ca admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:58.740478 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1393/2798] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:43:58 2022] PUT /identity/v3/projects/8c1583b00d9d4cb79051c90ca8501244/users/33fa0ec3fffe42bab594f1d8faf0fd76/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:43:58.751676 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf940e1-1827-4ae9-b747-24966ad9dd39 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:58.751950 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf940e1-1827-4ae9-b747-24966ad9dd39 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:58.752235 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf940e1-1827-4ae9-b747-24966ad9dd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:58.778389 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cdf940e1-1827-4ae9-b747-24966ad9dd39 None None] MFA Rules not processed for user `33fa0ec3fffe42bab594f1d8faf0fd76`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:43:58.779564 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cdf940e1-1827-4ae9-b747-24966ad9dd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:43:58.853550 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1406/2799] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:43:58 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:43:58.865429 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc0030-ca1b-42b6-b40a-b05e188291ca admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:58.867334 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b4bc0030-ca1b-42b6-b40a-b05e188291ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:43:58.930011 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc0030-ca1b-42b6-b40a-b05e188291ca admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:58.934386 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f593378-190a-4543-a705-71cbc04bfb12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:58.935438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f593378-190a-4543-a705-71cbc04bfb12 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:58.935438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f593378-190a-4543-a705-71cbc04bfb12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:58.935672 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f593378-190a-4543-a705-71cbc04bfb12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:58.945276 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f593378-190a-4543-a705-71cbc04bfb12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:58.946766 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f593378-190a-4543-a705-71cbc04bfb12 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:58.958276 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1394/2800] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:43:58 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:43:59.064879 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf940e1-1827-4ae9-b747-24966ad9dd39 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:43:59.075043 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf940e1-1827-4ae9-b747-24966ad9dd39 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:43:59.078738 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c78d2f-9ea8-403b-b39f-4663473d1892 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:43:59.079391 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c78d2f-9ea8-403b-b39f-4663473d1892 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:43:59.079683 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c78d2f-9ea8-403b-b39f-4663473d1892 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:43:59.080016 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c78d2f-9ea8-403b-b39f-4663473d1892 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:43:59.141893 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c78d2f-9ea8-403b-b39f-4663473d1892 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:43:59.143386 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c78d2f-9ea8-403b-b39f-4663473d1892 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:43:59.156377 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1407/2801] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:43:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:44:01.731236 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f593378-190a-4543-a705-71cbc04bfb12 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:01.792293 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f593378-190a-4543-a705-71cbc04bfb12 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:01.796857 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a436ba-638e-440b-9e40-775d399fce4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:01.797725 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a436ba-638e-440b-9e40-775d399fce4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:01.798035 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a436ba-638e-440b-9e40-775d399fce4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:01.798326 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a436ba-638e-440b-9e40-775d399fce4c admin admin] PATH_INFO: `/v3/users/73f0cada5a124f36b1b0a8eb36bc85c2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:01.799538 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a436ba-638e-440b-9e40-775d399fce4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=73f0cada5a124f36b1b0a8eb36bc85c2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:01.801447 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a436ba-638e-440b-9e40-775d399fce4c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:01.847967 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c4a436ba-638e-440b-9e40-775d399fce4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73f0cada5a124f36b1b0a8eb36bc85c2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:01.855999 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c4a436ba-638e-440b-9e40-775d399fce4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73f0cada5a124f36b1b0a8eb36bc85c2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:01.868072 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c4a436ba-638e-440b-9e40-775d399fce4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73f0cada5a124f36b1b0a8eb36bc85c2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:01.868704 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c4a436ba-638e-440b-9e40-775d399fce4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73f0cada5a124f36b1b0a8eb36bc85c2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:01.869309 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c4a436ba-638e-440b-9e40-775d399fce4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73f0cada5a124f36b1b0a8eb36bc85c2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:01.883962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1395/2802] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:44:01 2022] DELETE /identity/v3/users/73f0cada5a124f36b1b0a8eb36bc85c2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:44:01.888724 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cc094-459f-4045-ac4d-3a2d8497cacf None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:01.889212 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cc094-459f-4045-ac4d-3a2d8497cacf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:01.889322 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cc094-459f-4045-ac4d-3a2d8497cacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:01.906659 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ca7cc094-459f-4045-ac4d-3a2d8497cacf None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:44:01.907841 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ca7cc094-459f-4045-ac4d-3a2d8497cacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:01.947835 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:44:01.968000 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1408/2803] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:44:01.977500 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a436ba-638e-440b-9e40-775d399fce4c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:01.979694 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c4a436ba-638e-440b-9e40-775d399fce4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:01.993084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:44:02.038096 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a436ba-638e-440b-9e40-775d399fce4c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:02.042845 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f77d7-1544-46f3-b5ac-bff6c433390f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:02.043492 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f77d7-1544-46f3-b5ac-bff6c433390f service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:02.043782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f77d7-1544-46f3-b5ac-bff6c433390f service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:02.044065 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f77d7-1544-46f3-b5ac-bff6c433390f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:02.052883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f77d7-1544-46f3-b5ac-bff6c433390f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:02.054307 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f77d7-1544-46f3-b5ac-bff6c433390f service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:02.063965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1396/2804] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:44:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:44:02.331626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7cc094-459f-4045-ac4d-3a2d8497cacf None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:02.333045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ca7cc094-459f-4045-ac4d-3a2d8497cacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:02.351897 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7cc094-459f-4045-ac4d-3a2d8497cacf None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:02.358440 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51a7b3-e8eb-478b-b553-a44ac1254a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:02.359333 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51a7b3-e8eb-478b-b553-a44ac1254a1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:02.359618 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51a7b3-e8eb-478b-b553-a44ac1254a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:02.359893 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc51a7b3-e8eb-478b-b553-a44ac1254a1c admin admin] PATH_INFO: `/v3/projects/b523ac024205409f82db89412748ae1b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:02.361117 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51a7b3-e8eb-478b-b553-a44ac1254a1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b523ac024205409f82db89412748ae1b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:02.363121 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc51a7b3-e8eb-478b-b553-a44ac1254a1c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:02.412833 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cc51a7b3-e8eb-478b-b553-a44ac1254a1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b523ac024205409f82db89412748ae1b', 'request_id': 'req-cc51a7b3-e8eb-478b-b553-a44ac1254a1c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:02.421078 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cc51a7b3-e8eb-478b-b553-a44ac1254a1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b523ac024205409f82db89412748ae1b', 'request_id': 'req-cc51a7b3-e8eb-478b-b553-a44ac1254a1c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:02.427318 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cc51a7b3-e8eb-478b-b553-a44ac1254a1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b523ac024205409f82db89412748ae1b', 'request_id': 'req-cc51a7b3-e8eb-478b-b553-a44ac1254a1c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:02.438026 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cc51a7b3-e8eb-478b-b553-a44ac1254a1c admin admin] The token cache is being invalidate because project b523ac024205409f82db89412748ae1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:44:02.438323 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cc51a7b3-e8eb-478b-b553-a44ac1254a1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:02.439062 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cc51a7b3-e8eb-478b-b553-a44ac1254a1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:02.440684 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1409/2805] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:44:02 2022] DELETE /identity/v3/projects/b523ac024205409f82db89412748ae1b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:44:02.444661 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f77d7-1544-46f3-b5ac-bff6c433390f service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:02.455354 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f77d7-1544-46f3-b5ac-bff6c433390f service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:02.517381 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d676e-975a-4405-9cee-a44131081110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:02.518316 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d676e-975a-4405-9cee-a44131081110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:02.518695 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d676e-975a-4405-9cee-a44131081110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:02.518961 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d676e-975a-4405-9cee-a44131081110 admin admin] PATH_INFO: `/v3/users/33fa0ec3fffe42bab594f1d8faf0fd76` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:02.520382 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d676e-975a-4405-9cee-a44131081110 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33fa0ec3fffe42bab594f1d8faf0fd76)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:02.522629 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d676e-975a-4405-9cee-a44131081110 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:02.575039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-943d676e-975a-4405-9cee-a44131081110 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33fa0ec3fffe42bab594f1d8faf0fd76'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:02.584494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-943d676e-975a-4405-9cee-a44131081110 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33fa0ec3fffe42bab594f1d8faf0fd76'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:02.598927 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-943d676e-975a-4405-9cee-a44131081110 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33fa0ec3fffe42bab594f1d8faf0fd76'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:02.599603 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-943d676e-975a-4405-9cee-a44131081110 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33fa0ec3fffe42bab594f1d8faf0fd76'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:02.600271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-943d676e-975a-4405-9cee-a44131081110 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33fa0ec3fffe42bab594f1d8faf0fd76'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:02.612660 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1397/2806] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:44:02 2022] DELETE /identity/v3/users/33fa0ec3fffe42bab594f1d8faf0fd76 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:44:02.888219 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51a7b3-e8eb-478b-b553-a44ac1254a1c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:02.889701 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cc51a7b3-e8eb-478b-b553-a44ac1254a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:02.904592 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc51a7b3-e8eb-478b-b553-a44ac1254a1c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:02.964499 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fd88ce-dae4-4a59-8646-ac1447ad2028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:02.965360 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd88ce-dae4-4a59-8646-ac1447ad2028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:02.965658 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd88ce-dae4-4a59-8646-ac1447ad2028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:02.965916 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fd88ce-dae4-4a59-8646-ac1447ad2028 admin admin] PATH_INFO: `/v3/projects/8c1583b00d9d4cb79051c90ca8501244` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:02.967093 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd88ce-dae4-4a59-8646-ac1447ad2028 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c1583b00d9d4cb79051c90ca8501244)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:02.968969 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fd88ce-dae4-4a59-8646-ac1447ad2028 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:03.014285 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-94fd88ce-dae4-4a59-8646-ac1447ad2028 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c1583b00d9d4cb79051c90ca8501244', 'request_id': 'req-94fd88ce-dae4-4a59-8646-ac1447ad2028'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:03.021370 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-94fd88ce-dae4-4a59-8646-ac1447ad2028 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c1583b00d9d4cb79051c90ca8501244', 'request_id': 'req-94fd88ce-dae4-4a59-8646-ac1447ad2028'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:03.027776 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-94fd88ce-dae4-4a59-8646-ac1447ad2028 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c1583b00d9d4cb79051c90ca8501244', 'request_id': 'req-94fd88ce-dae4-4a59-8646-ac1447ad2028'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:03.034137 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:44:03.039413 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-94fd88ce-dae4-4a59-8646-ac1447ad2028 admin admin] The token cache is being invalidate because project 8c1583b00d9d4cb79051c90ca8501244 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:44:03.039749 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-94fd88ce-dae4-4a59-8646-ac1447ad2028 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:03.040700 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-94fd88ce-dae4-4a59-8646-ac1447ad2028 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:03.043454 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1410/2807] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:44:02 2022] DELETE /identity/v3/projects/8c1583b00d9d4cb79051c90ca8501244 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:44:03.049080 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e9380-4ce5-412f-8393-8a97d86452d5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:03.049561 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e9380-4ce5-412f-8393-8a97d86452d5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:03.049661 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e9380-4ce5-412f-8393-8a97d86452d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:03.066736 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-155e9380-4ce5-412f-8393-8a97d86452d5 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:44:03.067914 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-155e9380-4ce5-412f-8393-8a97d86452d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:03.099046 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:44:03.130929 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1398/2808] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:44:03.139675 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326334c7-f750-4c1f-b9c3-8be24277df8e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:03.139971 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326334c7-f750-4c1f-b9c3-8be24277df8e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:03.140265 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326334c7-f750-4c1f-b9c3-8be24277df8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:03.159087 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-326334c7-f750-4c1f-b9c3-8be24277df8e None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:44:03.160282 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-326334c7-f750-4c1f-b9c3-8be24277df8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:03.168281 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1411/2809] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:44:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:44:03.172311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e9380-4ce5-412f-8393-8a97d86452d5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:03.181818 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e9380-4ce5-412f-8393-8a97d86452d5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:03.186238 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd668cd9-ce94-4003-8c78-9b2ac393df89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:03.186922 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd668cd9-ce94-4003-8c78-9b2ac393df89 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:03.187201 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd668cd9-ce94-4003-8c78-9b2ac393df89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:03.187473 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd668cd9-ce94-4003-8c78-9b2ac393df89 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:03.188189 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd668cd9-ce94-4003-8c78-9b2ac393df89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:44:03.188494 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd668cd9-ce94-4003-8c78-9b2ac393df89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:03.190224 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd668cd9-ce94-4003-8c78-9b2ac393df89 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:03.202627 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1399/2810] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:44:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:44:03.206745 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326334c7-f750-4c1f-b9c3-8be24277df8e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:03.216456 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326334c7-f750-4c1f-b9c3-8be24277df8e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:03.221448 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65d1fa-5599-4586-8e48-f94ce0cb1cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:03.222125 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65d1fa-5599-4586-8e48-f94ce0cb1cee admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:03.222414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65d1fa-5599-4586-8e48-f94ce0cb1cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:03.222722 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65d1fa-5599-4586-8e48-f94ce0cb1cee admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:03.223405 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65d1fa-5599-4586-8e48-f94ce0cb1cee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:03.225278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65d1fa-5599-4586-8e48-f94ce0cb1cee admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:03.251121 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1412/2811] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:44:03 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:44:03.255318 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd668cd9-ce94-4003-8c78-9b2ac393df89 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:03.266782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd668cd9-ce94-4003-8c78-9b2ac393df89 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:03.324671 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502c780-ba01-4dcf-b410-0ea7e4d86f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:03.325348 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502c780-ba01-4dcf-b410-0ea7e4d86f2e admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:03.325633 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502c780-ba01-4dcf-b410-0ea7e4d86f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:03.326036 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a502c780-ba01-4dcf-b410-0ea7e4d86f2e admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:03.326857 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a502c780-ba01-4dcf-b410-0ea7e4d86f2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:03.328745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a502c780-ba01-4dcf-b410-0ea7e4d86f2e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:03.348536 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1400/2812] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:44:03 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 29 12:44:03.352531 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65d1fa-5599-4586-8e48-f94ce0cb1cee admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:03.362798 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65d1fa-5599-4586-8e48-f94ce0cb1cee admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:03.368027 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e67c7b-ef2e-4f98-b2a4-be4b0e404d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:03.368686 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e67c7b-ef2e-4f98-b2a4-be4b0e404d1f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:03.368972 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e67c7b-ef2e-4f98-b2a4-be4b0e404d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:03.369245 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e67c7b-ef2e-4f98-b2a4-be4b0e404d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:03.370501 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e67c7b-ef2e-4f98-b2a4-be4b0e404d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:03.372086 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e67c7b-ef2e-4f98-b2a4-be4b0e404d1f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:03.383526 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1413/2813] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:44:03.388236 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502c780-ba01-4dcf-b410-0ea7e4d86f2e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:03.399136 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a502c780-ba01-4dcf-b410-0ea7e4d86f2e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:03.404431 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7076d-3dad-4d87-83c3-93bb28c97779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:03.405083 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7076d-3dad-4d87-83c3-93bb28c97779 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:03.405363 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7076d-3dad-4d87-83c3-93bb28c97779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:03.405639 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7076d-3dad-4d87-83c3-93bb28c97779 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:03.406461 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7076d-3dad-4d87-83c3-93bb28c97779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:03.408039 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7076d-3dad-4d87-83c3-93bb28c97779 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:03.418876 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1401/2814] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:44:03.422867 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e67c7b-ef2e-4f98-b2a4-be4b0e404d1f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:03.432779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e67c7b-ef2e-4f98-b2a4-be4b0e404d1f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:03.437459 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fefc172-1d64-4f0c-aaf6-c4e4251897dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:03.438333 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fefc172-1d64-4f0c-aaf6-c4e4251897dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:03.438657 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fefc172-1d64-4f0c-aaf6-c4e4251897dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:03.438940 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fefc172-1d64-4f0c-aaf6-c4e4251897dd admin admin] PATH_INFO: `/v3/projects/e55c4975cfe0463f97a77b368b2dc97e/users/27e5c3293178435485a15041a567b5b7/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:03.453301 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fefc172-1d64-4f0c-aaf6-c4e4251897dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e55c4975cfe0463f97a77b368b2dc97e, user_id=27e5c3293178435485a15041a567b5b7, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:03.455974 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fefc172-1d64-4f0c-aaf6-c4e4251897dd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:03.468858 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1414/2815] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:44:03 2022] PUT /identity/v3/projects/e55c4975cfe0463f97a77b368b2dc97e/users/27e5c3293178435485a15041a567b5b7/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:44:03.480675 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65d5e5-c5f7-49d7-8660-49d99d841469 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:03.480965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65d5e5-c5f7-49d7-8660-49d99d841469 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:03.481244 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa65d5e5-c5f7-49d7-8660-49d99d841469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:03.507252 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-fa65d5e5-c5f7-49d7-8660-49d99d841469 None None] MFA Rules not processed for user `27e5c3293178435485a15041a567b5b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:44:03.508467 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fa65d5e5-c5f7-49d7-8660-49d99d841469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:03.571746 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1402/2816] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:44:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:44:03.582635 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fefc172-1d64-4f0c-aaf6-c4e4251897dd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:03.584284 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9fefc172-1d64-4f0c-aaf6-c4e4251897dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:03.643941 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fefc172-1d64-4f0c-aaf6-c4e4251897dd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:03.647800 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45ef08-ec24-4985-b493-082d70008f7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '356d396ae5cf47c3bc61f24f9e6dfa02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:03.648452 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45ef08-ec24-4985-b493-082d70008f7e service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:03.648735 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45ef08-ec24-4985-b493-082d70008f7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:03.649011 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45ef08-ec24-4985-b493-082d70008f7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:03.658220 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45ef08-ec24-4985-b493-082d70008f7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:03.659688 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45ef08-ec24-4985-b493-082d70008f7e service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:03.669971 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1415/2817] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:44:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:44:06.212636 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa65d5e5-c5f7-49d7-8660-49d99d841469 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:06.244182 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:44:06.278634 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa65d5e5-c5f7-49d7-8660-49d99d841469 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:06.283368 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6374a06-2bc9-4a2b-b4ee-15bb991b41fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:06.284279 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6374a06-2bc9-4a2b-b4ee-15bb991b41fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:06.284565 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6374a06-2bc9-4a2b-b4ee-15bb991b41fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:06.284840 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6374a06-2bc9-4a2b-b4ee-15bb991b41fe admin admin] PATH_INFO: `/v3/users/27e5c3293178435485a15041a567b5b7` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:06.286084 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6374a06-2bc9-4a2b-b4ee-15bb991b41fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=27e5c3293178435485a15041a567b5b7)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:06.288026 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6374a06-2bc9-4a2b-b4ee-15bb991b41fe admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:06.334466 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e6374a06-2bc9-4a2b-b4ee-15bb991b41fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27e5c3293178435485a15041a567b5b7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:06.341760 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e6374a06-2bc9-4a2b-b4ee-15bb991b41fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27e5c3293178435485a15041a567b5b7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:06.353819 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e6374a06-2bc9-4a2b-b4ee-15bb991b41fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27e5c3293178435485a15041a567b5b7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:06.354510 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e6374a06-2bc9-4a2b-b4ee-15bb991b41fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27e5c3293178435485a15041a567b5b7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:06.355123 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e6374a06-2bc9-4a2b-b4ee-15bb991b41fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27e5c3293178435485a15041a567b5b7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:06.368075 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1403/2818] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:44:06 2022] DELETE /identity/v3/users/27e5c3293178435485a15041a567b5b7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:44:06.373302 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c5012-dfd8-45ea-aa0b-bcc47711bfb0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:06.373774 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c5012-dfd8-45ea-aa0b-bcc47711bfb0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:06.373876 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c5012-dfd8-45ea-aa0b-bcc47711bfb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:06.391708 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-276c5012-dfd8-45ea-aa0b-bcc47711bfb0 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:44:06.392906 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-276c5012-dfd8-45ea-aa0b-bcc47711bfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:06.452414 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1416/2819] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:44:06.461998 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6374a06-2bc9-4a2b-b4ee-15bb991b41fe admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:06.463421 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e6374a06-2bc9-4a2b-b4ee-15bb991b41fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:06.519331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6374a06-2bc9-4a2b-b4ee-15bb991b41fe admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:06.523278 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da5cab-bdff-4f2b-a486-59f647bae95e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:06.524962 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da5cab-bdff-4f2b-a486-59f647bae95e service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:06.525248 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da5cab-bdff-4f2b-a486-59f647bae95e service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:06.525529 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da5cab-bdff-4f2b-a486-59f647bae95e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:06.534745 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da5cab-bdff-4f2b-a486-59f647bae95e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:06.536173 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da5cab-bdff-4f2b-a486-59f647bae95e service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:06.546230 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1404/2820] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:44:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:44:06.807756 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c5012-dfd8-45ea-aa0b-bcc47711bfb0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:06.809394 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-276c5012-dfd8-45ea-aa0b-bcc47711bfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:06.826580 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276c5012-dfd8-45ea-aa0b-bcc47711bfb0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:06.831336 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8cc5bc-45d3-4f70-a354-17270e2d67d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:06.832257 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cc5bc-45d3-4f70-a354-17270e2d67d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:06.832560 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cc5bc-45d3-4f70-a354-17270e2d67d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:06.832824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8cc5bc-45d3-4f70-a354-17270e2d67d8 admin admin] PATH_INFO: `/v3/projects/e55c4975cfe0463f97a77b368b2dc97e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:06.834085 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8cc5bc-45d3-4f70-a354-17270e2d67d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e55c4975cfe0463f97a77b368b2dc97e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:06.836079 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8cc5bc-45d3-4f70-a354-17270e2d67d8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:06.872394 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:44:06.886229 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9e8cc5bc-45d3-4f70-a354-17270e2d67d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e55c4975cfe0463f97a77b368b2dc97e', 'request_id': 'req-9e8cc5bc-45d3-4f70-a354-17270e2d67d8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:06.893085 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9e8cc5bc-45d3-4f70-a354-17270e2d67d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e55c4975cfe0463f97a77b368b2dc97e', 'request_id': 'req-9e8cc5bc-45d3-4f70-a354-17270e2d67d8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:06.899655 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9e8cc5bc-45d3-4f70-a354-17270e2d67d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e55c4975cfe0463f97a77b368b2dc97e', 'request_id': 'req-9e8cc5bc-45d3-4f70-a354-17270e2d67d8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:06.910736 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9e8cc5bc-45d3-4f70-a354-17270e2d67d8 admin admin] The token cache is being invalidate because project e55c4975cfe0463f97a77b368b2dc97e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:44:06.911026 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9e8cc5bc-45d3-4f70-a354-17270e2d67d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:06.911665 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9e8cc5bc-45d3-4f70-a354-17270e2d67d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:06.913194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1417/2821] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:44:06 2022] DELETE /identity/v3/projects/e55c4975cfe0463f97a77b368b2dc97e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:44:06.918907 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fa95f-44d9-4969-83af-ec34ff770e04 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:06.919200 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fa95f-44d9-4969-83af-ec34ff770e04 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:06.919485 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fa95f-44d9-4969-83af-ec34ff770e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:06.940813 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-353fa95f-44d9-4969-83af-ec34ff770e04 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:44:06.942014 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-353fa95f-44d9-4969-83af-ec34ff770e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:06.986728 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:44:07.001972 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1405/2822] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:44:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:44:07.009874 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0242e4-7f42-4c51-9e32-833e54856d69 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:07.010176 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0242e4-7f42-4c51-9e32-833e54856d69 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:07.010498 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0242e4-7f42-4c51-9e32-833e54856d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:07.029476 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8a0242e4-7f42-4c51-9e32-833e54856d69 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:44:07.030725 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8a0242e4-7f42-4c51-9e32-833e54856d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:07.038675 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1418/2823] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:44:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:44:07.042698 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fa95f-44d9-4969-83af-ec34ff770e04 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:07.053408 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fa95f-44d9-4969-83af-ec34ff770e04 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:07.057947 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b671d-27b6-414a-ac7d-5e433dcc44f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:07.059027 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b671d-27b6-414a-ac7d-5e433dcc44f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:07.059322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b671d-27b6-414a-ac7d-5e433dcc44f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:07.059608 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b671d-27b6-414a-ac7d-5e433dcc44f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:07.060367 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b671d-27b6-414a-ac7d-5e433dcc44f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:44:07.060678 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b671d-27b6-414a-ac7d-5e433dcc44f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:07.062271 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b671d-27b6-414a-ac7d-5e433dcc44f3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:07.073776 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1406/2824] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:44:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:44:07.078568 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0242e4-7f42-4c51-9e32-833e54856d69 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:07.087705 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0242e4-7f42-4c51-9e32-833e54856d69 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:07.092893 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d105be1-8604-4cf1-aa06-7d8dbe0d9632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:07.093535 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d105be1-8604-4cf1-aa06-7d8dbe0d9632 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:07.093821 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d105be1-8604-4cf1-aa06-7d8dbe0d9632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:07.094122 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d105be1-8604-4cf1-aa06-7d8dbe0d9632 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:07.094846 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d105be1-8604-4cf1-aa06-7d8dbe0d9632 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:07.096728 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d105be1-8604-4cf1-aa06-7d8dbe0d9632 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:07.125884 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1419/2825] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:44:07 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:44:07.127393 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b671d-27b6-414a-ac7d-5e433dcc44f3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:07.138532 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b671d-27b6-414a-ac7d-5e433dcc44f3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:07.202002 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c75b8b-9eb4-46bb-902c-b9e64fc2cd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:07.202723 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c75b8b-9eb4-46bb-902c-b9e64fc2cd4c admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:07.203016 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c75b8b-9eb4-46bb-902c-b9e64fc2cd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:07.203319 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c75b8b-9eb4-46bb-902c-b9e64fc2cd4c admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:07.204002 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c75b8b-9eb4-46bb-902c-b9e64fc2cd4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:07.205965 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c75b8b-9eb4-46bb-902c-b9e64fc2cd4c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:07.230147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1407/2826] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:44:07 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:44:07.234260 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d105be1-8604-4cf1-aa06-7d8dbe0d9632 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:07.245216 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d105be1-8604-4cf1-aa06-7d8dbe0d9632 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:07.249856 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d2a9b-f0d5-4671-ad62-3c9e1a452487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:07.250579 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d2a9b-f0d5-4671-ad62-3c9e1a452487 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:07.250866 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d2a9b-f0d5-4671-ad62-3c9e1a452487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:07.251146 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d2a9b-f0d5-4671-ad62-3c9e1a452487 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:07.251865 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d2a9b-f0d5-4671-ad62-3c9e1a452487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:07.253447 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d2a9b-f0d5-4671-ad62-3c9e1a452487 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:07.264283 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1420/2827] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:44:07.268231 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c75b8b-9eb4-46bb-902c-b9e64fc2cd4c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:07.279569 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c75b8b-9eb4-46bb-902c-b9e64fc2cd4c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:07.284887 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ca817-b784-46af-825b-a8cf7b1e2911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:07.285626 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ca817-b784-46af-825b-a8cf7b1e2911 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:07.285911 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ca817-b784-46af-825b-a8cf7b1e2911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:07.286315 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ca817-b784-46af-825b-a8cf7b1e2911 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:07.287234 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ca817-b784-46af-825b-a8cf7b1e2911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:07.288993 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ca817-b784-46af-825b-a8cf7b1e2911 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:07.299960 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1408/2828] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:44:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:44:07.304022 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d2a9b-f0d5-4671-ad62-3c9e1a452487 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:07.314802 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d2a9b-f0d5-4671-ad62-3c9e1a452487 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:07.320094 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2531db-e272-4ed0-96c2-e845f7ed1dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:07.320981 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2531db-e272-4ed0-96c2-e845f7ed1dd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:07.321269 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2531db-e272-4ed0-96c2-e845f7ed1dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:07.321555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2531db-e272-4ed0-96c2-e845f7ed1dd2 admin admin] PATH_INFO: `/v3/projects/925a9005d7494b04923bdaa194731362/users/fd70c574b180445f9348d9aee8ac4c82/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:07.337276 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2531db-e272-4ed0-96c2-e845f7ed1dd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=925a9005d7494b04923bdaa194731362, user_id=fd70c574b180445f9348d9aee8ac4c82, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:07.340666 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2531db-e272-4ed0-96c2-e845f7ed1dd2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:07.357450 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1421/2829] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:44:07 2022] PUT /identity/v3/projects/925a9005d7494b04923bdaa194731362/users/fd70c574b180445f9348d9aee8ac4c82/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:44:07.362750 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ac4e5-555a-49af-9ead-a7df6cc3db7c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:07.363046 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ac4e5-555a-49af-9ead-a7df6cc3db7c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:07.363438 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ac4e5-555a-49af-9ead-a7df6cc3db7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:07.381915 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8d5ac4e5-555a-49af-9ead-a7df6cc3db7c None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:44:07.383239 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8d5ac4e5-555a-49af-9ead-a7df6cc3db7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:07.443590 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1409/2830] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:44:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:44:07.454626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2531db-e272-4ed0-96c2-e845f7ed1dd2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:07.456402 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0f2531db-e272-4ed0-96c2-e845f7ed1dd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:07.513269 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2531db-e272-4ed0-96c2-e845f7ed1dd2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:07.516986 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb5de2-9ca9-4d4a-b5fa-de6054bd382c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:07.517626 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb5de2-9ca9-4d4a-b5fa-de6054bd382c service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:07.517917 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb5de2-9ca9-4d4a-b5fa-de6054bd382c service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:07.518196 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eb5de2-9ca9-4d4a-b5fa-de6054bd382c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:07.527579 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb5de2-9ca9-4d4a-b5fa-de6054bd382c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:07.529002 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eb5de2-9ca9-4d4a-b5fa-de6054bd382c service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:07.538295 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1422/2831] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:44:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:44:08.852953 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ac4e5-555a-49af-9ead-a7df6cc3db7c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:08.856021 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8d5ac4e5-555a-49af-9ead-a7df6cc3db7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:08.868696 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ac4e5-555a-49af-9ead-a7df6cc3db7c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:08.874638 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944e624-1d5a-4090-a5dd-0f7fcc67b70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:08.876105 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944e624-1d5a-4090-a5dd-0f7fcc67b70e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:08.876582 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944e624-1d5a-4090-a5dd-0f7fcc67b70e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:08.876960 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944e624-1d5a-4090-a5dd-0f7fcc67b70e admin admin] PATH_INFO: `/v3/users/eb9bae5e3522437182d2650c3fcfdef6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:08.878741 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d944e624-1d5a-4090-a5dd-0f7fcc67b70e admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb9bae5e3522437182d2650c3fcfdef6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:08.882283 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d944e624-1d5a-4090-a5dd-0f7fcc67b70e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:08.943615 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d944e624-1d5a-4090-a5dd-0f7fcc67b70e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb9bae5e3522437182d2650c3fcfdef6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:08.949214 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:44:08.955437 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d944e624-1d5a-4090-a5dd-0f7fcc67b70e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb9bae5e3522437182d2650c3fcfdef6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:08.973919 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d944e624-1d5a-4090-a5dd-0f7fcc67b70e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb9bae5e3522437182d2650c3fcfdef6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:08.974731 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d944e624-1d5a-4090-a5dd-0f7fcc67b70e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb9bae5e3522437182d2650c3fcfdef6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:08.975535 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d944e624-1d5a-4090-a5dd-0f7fcc67b70e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb9bae5e3522437182d2650c3fcfdef6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:08.996791 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1410/2832] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:44:08 2022] DELETE /identity/v3/users/eb9bae5e3522437182d2650c3fcfdef6 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:44:09.219700 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb5de2-9ca9-4d4a-b5fa-de6054bd382c service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:09.231416 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eb5de2-9ca9-4d4a-b5fa-de6054bd382c service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:09.290788 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5567aca-ecba-4f30-a0ae-f4854e230fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:09.291687 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5567aca-ecba-4f30-a0ae-f4854e230fbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:09.292048 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5567aca-ecba-4f30-a0ae-f4854e230fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:09.292410 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5567aca-ecba-4f30-a0ae-f4854e230fbf admin admin] PATH_INFO: `/v3/projects/e1ba9e59fe574391a0c8342daed01416` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:09.294244 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5567aca-ecba-4f30-a0ae-f4854e230fbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1ba9e59fe574391a0c8342daed01416)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:09.296262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5567aca-ecba-4f30-a0ae-f4854e230fbf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:09.297873 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:44:09.356904 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a5567aca-ecba-4f30-a0ae-f4854e230fbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1ba9e59fe574391a0c8342daed01416', 'request_id': 'req-a5567aca-ecba-4f30-a0ae-f4854e230fbf'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:09.368690 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a5567aca-ecba-4f30-a0ae-f4854e230fbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1ba9e59fe574391a0c8342daed01416', 'request_id': 'req-a5567aca-ecba-4f30-a0ae-f4854e230fbf'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:09.375527 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a5567aca-ecba-4f30-a0ae-f4854e230fbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1ba9e59fe574391a0c8342daed01416', 'request_id': 'req-a5567aca-ecba-4f30-a0ae-f4854e230fbf'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:09.387747 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a5567aca-ecba-4f30-a0ae-f4854e230fbf admin admin] The token cache is being invalidate because project e1ba9e59fe574391a0c8342daed01416 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:44:09.388141 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a5567aca-ecba-4f30-a0ae-f4854e230fbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:09.388877 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a5567aca-ecba-4f30-a0ae-f4854e230fbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:44:09.390675 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1423/2833] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:44:09 2022] DELETE /identity/v3/projects/e1ba9e59fe574391a0c8342daed01416 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:44:11.653577 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac26c7a-5bc0-46f5-840d-2b9278012b5c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:11.654077 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac26c7a-5bc0-46f5-840d-2b9278012b5c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:11.654196 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac26c7a-5bc0-46f5-840d-2b9278012b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:11.684986 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6ac26c7a-5bc0-46f5-840d-2b9278012b5c None None] MFA Rules not processed for user `fd70c574b180445f9348d9aee8ac4c82`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:44:11.686192 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6ac26c7a-5bc0-46f5-840d-2b9278012b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:11.753429 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1411/2834] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:44:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:44:11.757927 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5567aca-ecba-4f30-a0ae-f4854e230fbf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:11.759305 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a5567aca-ecba-4f30-a0ae-f4854e230fbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:11.825479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5567aca-ecba-4f30-a0ae-f4854e230fbf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:11.830330 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdcce8-cdef-453e-9d6b-33a89cbe7571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:11.831032 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdcce8-cdef-453e-9d6b-33a89cbe7571 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:11.831318 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdcce8-cdef-453e-9d6b-33a89cbe7571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:11.831595 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdcce8-cdef-453e-9d6b-33a89cbe7571 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:11.832312 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cdcce8-cdef-453e-9d6b-33a89cbe7571 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:11.834206 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cdcce8-cdef-453e-9d6b-33a89cbe7571 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:11.863188 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1424/2835] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:44:11 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:44:11.867268 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac26c7a-5bc0-46f5-840d-2b9278012b5c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:11.879068 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac26c7a-5bc0-46f5-840d-2b9278012b5c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:11.935378 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e78c5-34fa-40bb-ab62-9430a0f00611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:11.936046 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e78c5-34fa-40bb-ab62-9430a0f00611 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:11.936346 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e78c5-34fa-40bb-ab62-9430a0f00611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:11.936604 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e78c5-34fa-40bb-ab62-9430a0f00611 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:11.937298 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e78c5-34fa-40bb-ab62-9430a0f00611 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:11.939236 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e78c5-34fa-40bb-ab62-9430a0f00611 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:11.958264 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1412/2836] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:44:11 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:44:11.962666 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdcce8-cdef-453e-9d6b-33a89cbe7571 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:11.973956 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdcce8-cdef-453e-9d6b-33a89cbe7571 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:11.978633 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da60fc95-ff10-45e1-ad2e-d8ae695302d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:11.979298 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60fc95-ff10-45e1-ad2e-d8ae695302d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:11.979589 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60fc95-ff10-45e1-ad2e-d8ae695302d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:11.979872 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60fc95-ff10-45e1-ad2e-d8ae695302d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:11.981446 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da60fc95-ff10-45e1-ad2e-d8ae695302d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:11.983082 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da60fc95-ff10-45e1-ad2e-d8ae695302d8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:11.992889 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1425/2837] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:44:11.997045 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e78c5-34fa-40bb-ab62-9430a0f00611 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:12.007113 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e78c5-34fa-40bb-ab62-9430a0f00611 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:12.011930 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5b22e-80c6-4b75-a016-e3a4fae4462c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:12.012731 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5b22e-80c6-4b75-a016-e3a4fae4462c admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:12.013016 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5b22e-80c6-4b75-a016-e3a4fae4462c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:12.013298 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5b22e-80c6-4b75-a016-e3a4fae4462c admin admin] PATH_INFO: `/v3/projects/568410b0a79d457285548e6e78cbed6c/users/850d4a3d03434a9dac9814f5b4a70db4/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:12.028528 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5b22e-80c6-4b75-a016-e3a4fae4462c admin admin] RBAC: Authorizing `identity:create_grant(project_id=568410b0a79d457285548e6e78cbed6c, user_id=850d4a3d03434a9dac9814f5b4a70db4, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:12.031222 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5b22e-80c6-4b75-a016-e3a4fae4462c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:12.045317 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1413/2838] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:44:11 2022] PUT /identity/v3/projects/568410b0a79d457285548e6e78cbed6c/users/850d4a3d03434a9dac9814f5b4a70db4/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:44:12.049248 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da60fc95-ff10-45e1-ad2e-d8ae695302d8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:12.059906 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da60fc95-ff10-45e1-ad2e-d8ae695302d8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:12.115099 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c30b75-a142-4871-b96c-67f0b97227f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:12.115760 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c30b75-a142-4871-b96c-67f0b97227f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:12.116045 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c30b75-a142-4871-b96c-67f0b97227f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:12.116317 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c30b75-a142-4871-b96c-67f0b97227f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:12.117030 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c30b75-a142-4871-b96c-67f0b97227f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:12.118630 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c30b75-a142-4871-b96c-67f0b97227f5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:12.124318 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:44:12.129519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1426/2839] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:44:12.133637 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5b22e-80c6-4b75-a016-e3a4fae4462c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:12.144596 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5b22e-80c6-4b75-a016-e3a4fae4462c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:12.149414 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa40012-c133-4f10-a440-997b12188dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:12.150287 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa40012-c133-4f10-a440-997b12188dce admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:12.150610 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa40012-c133-4f10-a440-997b12188dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:12.150891 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa40012-c133-4f10-a440-997b12188dce admin admin] PATH_INFO: `/v3/projects/568410b0a79d457285548e6e78cbed6c/users/850d4a3d03434a9dac9814f5b4a70db4/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:12.152979 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa40012-c133-4f10-a440-997b12188dce admin admin] RBAC: Authorizing `identity:create_grant(project_id=568410b0a79d457285548e6e78cbed6c, user_id=850d4a3d03434a9dac9814f5b4a70db4, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:12.155659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa40012-c133-4f10-a440-997b12188dce admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:12.165735 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1414/2840] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:44:12 2022] PUT /identity/v3/projects/568410b0a79d457285548e6e78cbed6c/users/850d4a3d03434a9dac9814f5b4a70db4/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:44:16.773748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353a443-ee2a-4c6b-a24d-de2435e41f13 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:16.774857 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353a443-ee2a-4c6b-a24d-de2435e41f13 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:16.775383 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d353a443-ee2a-4c6b-a24d-de2435e41f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:16.814872 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d353a443-ee2a-4c6b-a24d-de2435e41f13 None None] MFA Rules not processed for user `850d4a3d03434a9dac9814f5b4a70db4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:44:16.816125 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d353a443-ee2a-4c6b-a24d-de2435e41f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:16.918895 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1427/2841] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:44:16 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:44:16.932311 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa40012-c133-4f10-a440-997b12188dce admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:16.934543 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-baa40012-c133-4f10-a440-997b12188dce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:16.962231 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:44:17.029170 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa40012-c133-4f10-a440-997b12188dce admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:17.033589 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6bde3c-9c19-456f-9507-8b941bf98f99 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '096234e9f15541c0b187dc8dd527b0fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:17.034272 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6bde3c-9c19-456f-9507-8b941bf98f99 service glance] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:17.034594 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6bde3c-9c19-456f-9507-8b941bf98f99 service glance] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:17.034874 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6bde3c-9c19-456f-9507-8b941bf98f99 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:17.110034 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6bde3c-9c19-456f-9507-8b941bf98f99 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:17.111646 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6bde3c-9c19-456f-9507-8b941bf98f99 service glance] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:17.127350 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1415/2842] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:44:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:44:17.247477 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349dc96-4869-497e-bb03-9ce47d623300 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:17.247780 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349dc96-4869-497e-bb03-9ce47d623300 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:17.248060 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349dc96-4869-497e-bb03-9ce47d623300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:17.273631 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b349dc96-4869-497e-bb03-9ce47d623300 None None] MFA Rules not processed for user `8c10852a728844c6ae0a48e3772d95b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:44:17.275001 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b349dc96-4869-497e-bb03-9ce47d623300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:17.332647 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1428/2843] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jun 29 12:44:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:44:17.389407 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6bde3c-9c19-456f-9507-8b941bf98f99 service glance] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:17.390840 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6d6bde3c-9c19-456f-9507-8b941bf98f99 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:17.461574 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6bde3c-9c19-456f-9507-8b941bf98f99 service glance] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:17.466499 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c856e-70e7-4395-84a1-dae95d76df2d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209b0df45f76404da880a9ae0c637d67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:17.467145 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c856e-70e7-4395-84a1-dae95d76df2d service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:17.467429 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c856e-70e7-4395-84a1-dae95d76df2d service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:17.467704 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c856e-70e7-4395-84a1-dae95d76df2d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:17.476406 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c856e-70e7-4395-84a1-dae95d76df2d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:17.477997 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c856e-70e7-4395-84a1-dae95d76df2d service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:17.486842 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1416/2844] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 12:44:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 12:44:26.016191 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0493e4-0d41-44f8-a86c-69f2396343b0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:26.016718 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0493e4-0d41-44f8-a86c-69f2396343b0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:26.016821 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0493e4-0d41-44f8-a86c-69f2396343b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:26.034778 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ca0493e4-0d41-44f8-a86c-69f2396343b0 None None] MFA Rules not processed for user `fd70c574b180445f9348d9aee8ac4c82`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:44:26.035959 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ca0493e4-0d41-44f8-a86c-69f2396343b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:26.042895 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1429/2845] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:44:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:44:26.611827 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c856e-70e7-4395-84a1-dae95d76df2d service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:26.613366 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3b6c856e-70e7-4395-84a1-dae95d76df2d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:26.673020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:44:26.688942 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c856e-70e7-4395-84a1-dae95d76df2d service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:26.693924 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50aae7d-6ae7-44cb-8a34-b4abdf8e7c1f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98135cacb9d4f22846ad33500b9bd1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:26.694630 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50aae7d-6ae7-44cb-8a34-b4abdf8e7c1f service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:26.694917 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50aae7d-6ae7-44cb-8a34-b4abdf8e7c1f service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:26.695198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50aae7d-6ae7-44cb-8a34-b4abdf8e7c1f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:26.696961 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f50aae7d-6ae7-44cb-8a34-b4abdf8e7c1f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:26.708348 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50aae7d-6ae7-44cb-8a34-b4abdf8e7c1f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:26.709912 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50aae7d-6ae7-44cb-8a34-b4abdf8e7c1f service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:26.719756 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1417/2846] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:44:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:44:28.301748 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55989b9b-10e4-4edb-bd4a-a3e44b703170 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:28.302269 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55989b9b-10e4-4edb-bd4a-a3e44b703170 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:28.302384 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55989b9b-10e4-4edb-bd4a-a3e44b703170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:28.325919 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-55989b9b-10e4-4edb-bd4a-a3e44b703170 None None] MFA Rules not processed for user `8c10852a728844c6ae0a48e3772d95b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:44:28.328241 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-55989b9b-10e4-4edb-bd4a-a3e44b703170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:28.338434 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1430/2847] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jun 29 12:44:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:44:28.352422 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50aae7d-6ae7-44cb-8a34-b4abdf8e7c1f service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:28.364732 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50aae7d-6ae7-44cb-8a34-b4abdf8e7c1f service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:28.369768 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d0c63-f5e8-48a4-919a-ef770c36e067 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209b0df45f76404da880a9ae0c637d67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:28.370414 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d0c63-f5e8-48a4-919a-ef770c36e067 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:28.371409 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d0c63-f5e8-48a4-919a-ef770c36e067 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:28.371711 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d0c63-f5e8-48a4-919a-ef770c36e067 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:28.380410 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d0c63-f5e8-48a4-919a-ef770c36e067 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:28.382013 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d0c63-f5e8-48a4-919a-ef770c36e067 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:28.390659 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1418/2848] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 12:44:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 12:44:36.443653 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55989b9b-10e4-4edb-bd4a-a3e44b703170 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:36.445256 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-55989b9b-10e4-4edb-bd4a-a3e44b703170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:36.504123 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55989b9b-10e4-4edb-bd4a-a3e44b703170 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:36.507653 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f6159-6f1c-459b-96ef-85fae95dce61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:36.508291 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f6159-6f1c-459b-96ef-85fae95dce61 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:36.508572 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f6159-6f1c-459b-96ef-85fae95dce61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:36.508845 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f6159-6f1c-459b-96ef-85fae95dce61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:36.510604 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-846f6159-6f1c-459b-96ef-85fae95dce61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:36.520277 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f6159-6f1c-459b-96ef-85fae95dce61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:36.521943 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f6159-6f1c-459b-96ef-85fae95dce61 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:36.530669 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1431/2849] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:44:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:44:37.160409 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d0c63-f5e8-48a4-919a-ef770c36e067 service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:37.161940 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-179d0c63-f5e8-48a4-919a-ef770c36e067 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:37.266956 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d0c63-f5e8-48a4-919a-ef770c36e067 service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:37.273561 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea681d-3dcd-4efe-86ba-08135c1ee822 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:37.274566 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea681d-3dcd-4efe-86ba-08135c1ee822 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:37.275063 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea681d-3dcd-4efe-86ba-08135c1ee822 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:37.275535 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ea681d-3dcd-4efe-86ba-08135c1ee822 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:37.277512 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b4ea681d-3dcd-4efe-86ba-08135c1ee822 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:37.291433 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ea681d-3dcd-4efe-86ba-08135c1ee822 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:37.294143 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ea681d-3dcd-4efe-86ba-08135c1ee822 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:37.305933 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1419/2850] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:44:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 12:44:42.893556 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f6159-6f1c-459b-96ef-85fae95dce61 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:42.901588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:44:42.908438 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f6159-6f1c-459b-96ef-85fae95dce61 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:42.914202 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1da955-096d-4042-93a5-c98cc9a287d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:42.914982 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1da955-096d-4042-93a5-c98cc9a287d2 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:42.915338 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1da955-096d-4042-93a5-c98cc9a287d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:42.915688 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1da955-096d-4042-93a5-c98cc9a287d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:42.917509 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ad1da955-096d-4042-93a5-c98cc9a287d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:42.930210 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1da955-096d-4042-93a5-c98cc9a287d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:42.931822 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1da955-096d-4042-93a5-c98cc9a287d2 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:42.943266 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1432/2851] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:44:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:44:47.300646 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=47 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:44:52.938505 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:44:53.537170 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea681d-3dcd-4efe-86ba-08135c1ee822 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:53.538592 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b4ea681d-3dcd-4efe-86ba-08135c1ee822 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:53.550319 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ea681d-3dcd-4efe-86ba-08135c1ee822 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:53.553828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab31c2dc-0b21-4b63-b1da-315f8bdb0ea9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:53.554477 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31c2dc-0b21-4b63-b1da-315f8bdb0ea9 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:53.554762 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31c2dc-0b21-4b63-b1da-315f8bdb0ea9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:53.555036 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab31c2dc-0b21-4b63-b1da-315f8bdb0ea9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:53.564910 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab31c2dc-0b21-4b63-b1da-315f8bdb0ea9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:53.566321 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab31c2dc-0b21-4b63-b1da-315f8bdb0ea9 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:53.575828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1420/2852] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:44:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:44:55.112839 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1da955-096d-4042-93a5-c98cc9a287d2 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:44:55.125322 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1da955-096d-4042-93a5-c98cc9a287d2 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:44:55.132170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9d5b7-9fd5-44d9-8da9-871793d9bb9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:44:55.133249 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9d5b7-9fd5-44d9-8da9-871793d9bb9d service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:44:55.133789 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9d5b7-9fd5-44d9-8da9-871793d9bb9d service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:44:55.134077 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea9d5b7-9fd5-44d9-8da9-871793d9bb9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:44:55.136048 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dea9d5b7-9fd5-44d9-8da9-871793d9bb9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:44:55.151299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea9d5b7-9fd5-44d9-8da9-871793d9bb9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:44:55.152904 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea9d5b7-9fd5-44d9-8da9-871793d9bb9d service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:44:55.166026 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1433/2853] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:44:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:45:03.571664 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:45:05.160571 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:46:15.915269 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbdcff-0e19-4ce7-b19a-1d0d3a46c7cd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:15.915802 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbdcff-0e19-4ce7-b19a-1d0d3a46c7cd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:15.915908 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbdcff-0e19-4ce7-b19a-1d0d3a46c7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:15.954052 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2bfbdcff-0e19-4ce7-b19a-1d0d3a46c7cd None None] MFA Rules not processed for user `8c10852a728844c6ae0a48e3772d95b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:46:15.955270 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2bfbdcff-0e19-4ce7-b19a-1d0d3a46c7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:46:15.988602 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1421/2854] 10.222.0.52 () {58 vars in 967 bytes} [Wed Jun 29 12:46:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:46:16.029950 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9d5b7-9fd5-44d9-8da9-871793d9bb9d service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:46:16.072764 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea9d5b7-9fd5-44d9-8da9-871793d9bb9d service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:46:16.077883 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df428c-8399-4b82-a4c3-f9a9d1e012aa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '209b0df45f76404da880a9ae0c637d67', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:46:16.078551 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df428c-8399-4b82-a4c3-f9a9d1e012aa service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:16.078837 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df428c-8399-4b82-a4c3-f9a9d1e012aa service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:16.079115 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1df428c-8399-4b82-a4c3-f9a9d1e012aa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:16.087234 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df428c-8399-4b82-a4c3-f9a9d1e012aa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:46:16.088792 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1df428c-8399-4b82-a4c3-f9a9d1e012aa service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:46:16.096762 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1434/2855] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jun 29 12:46:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 29 12:46:19.266286 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbdcff-0e19-4ce7-b19a-1d0d3a46c7cd None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:46:19.275468 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2bfbdcff-0e19-4ce7-b19a-1d0d3a46c7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:46:19.300279 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbdcff-0e19-4ce7-b19a-1d0d3a46c7cd None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:46:19.307573 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e01fa-1c1d-45ca-ad97-9a496d69236d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98135cacb9d4f22846ad33500b9bd1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:46:19.308424 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e01fa-1c1d-45ca-ad97-9a496d69236d service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:19.308895 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e01fa-1c1d-45ca-ad97-9a496d69236d service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:19.309349 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1e01fa-1c1d-45ca-ad97-9a496d69236d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:19.311411 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5a1e01fa-1c1d-45ca-ad97-9a496d69236d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:46:19.325328 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e01fa-1c1d-45ca-ad97-9a496d69236d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:46:19.329259 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1e01fa-1c1d-45ca-ad97-9a496d69236d service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:46:19.355946 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1422/2856] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:46:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 12:46:22.464520 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df428c-8399-4b82-a4c3-f9a9d1e012aa service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:46:22.476775 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1df428c-8399-4b82-a4c3-f9a9d1e012aa service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:46:22.481799 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bdd110-2e64-4397-aaaa-959a029a7a70 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98135cacb9d4f22846ad33500b9bd1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:46:22.482468 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bdd110-2e64-4397-aaaa-959a029a7a70 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:22.482764 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bdd110-2e64-4397-aaaa-959a029a7a70 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:22.483050 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bdd110-2e64-4397-aaaa-959a029a7a70 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:22.484812 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-19bdd110-2e64-4397-aaaa-959a029a7a70 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:46:22.496640 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bdd110-2e64-4397-aaaa-959a029a7a70 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:46:22.498282 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bdd110-2e64-4397-aaaa-959a029a7a70 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:46:22.508588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1435/2857] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:46:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:46:22.527479 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e01fa-1c1d-45ca-ad97-9a496d69236d service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:46:22.621950 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1e01fa-1c1d-45ca-ad97-9a496d69236d service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:46:22.626496 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbeaae7-881c-4536-bc06-80dbfec5f294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:46:22.627367 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbeaae7-881c-4536-bc06-80dbfec5f294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:22.627652 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbeaae7-881c-4536-bc06-80dbfec5f294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:22.627935 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbeaae7-881c-4536-bc06-80dbfec5f294 admin admin] PATH_INFO: `/v3/users/fd70c574b180445f9348d9aee8ac4c82` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:22.629111 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbeaae7-881c-4536-bc06-80dbfec5f294 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd70c574b180445f9348d9aee8ac4c82)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:46:22.631789 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbeaae7-881c-4536-bc06-80dbfec5f294 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:46:22.679941 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2dbeaae7-881c-4536-bc06-80dbfec5f294 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd70c574b180445f9348d9aee8ac4c82'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:46:22.686898 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2dbeaae7-881c-4536-bc06-80dbfec5f294 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd70c574b180445f9348d9aee8ac4c82'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:46:22.698863 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2dbeaae7-881c-4536-bc06-80dbfec5f294 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd70c574b180445f9348d9aee8ac4c82'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:46:22.699491 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2dbeaae7-881c-4536-bc06-80dbfec5f294 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd70c574b180445f9348d9aee8ac4c82'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:46:22.700088 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2dbeaae7-881c-4536-bc06-80dbfec5f294 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd70c574b180445f9348d9aee8ac4c82'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:46:22.707521 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:46:22.713579 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1423/2858] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:46:22 2022] DELETE /identity/v3/users/fd70c574b180445f9348d9aee8ac4c82 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:46:22.887670 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bdd110-2e64-4397-aaaa-959a029a7a70 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:46:22.898923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bdd110-2e64-4397-aaaa-959a029a7a70 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:46:22.960537 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83112bf5-9de2-429f-a834-495daafa1be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:46:22.961525 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83112bf5-9de2-429f-a834-495daafa1be2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:22.961813 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83112bf5-9de2-429f-a834-495daafa1be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:22.962090 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83112bf5-9de2-429f-a834-495daafa1be2 admin admin] PATH_INFO: `/v3/projects/925a9005d7494b04923bdaa194731362` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:22.965160 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83112bf5-9de2-429f-a834-495daafa1be2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=925a9005d7494b04923bdaa194731362)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:46:22.965767 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83112bf5-9de2-429f-a834-495daafa1be2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:46:23.017768 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-83112bf5-9de2-429f-a834-495daafa1be2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '925a9005d7494b04923bdaa194731362', 'request_id': 'req-83112bf5-9de2-429f-a834-495daafa1be2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:46:23.026128 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-83112bf5-9de2-429f-a834-495daafa1be2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '925a9005d7494b04923bdaa194731362', 'request_id': 'req-83112bf5-9de2-429f-a834-495daafa1be2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:46:23.032667 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-83112bf5-9de2-429f-a834-495daafa1be2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '925a9005d7494b04923bdaa194731362', 'request_id': 'req-83112bf5-9de2-429f-a834-495daafa1be2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:46:23.057752 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-83112bf5-9de2-429f-a834-495daafa1be2 admin admin] The token cache is being invalidate because project 925a9005d7494b04923bdaa194731362 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:46:23.058111 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-83112bf5-9de2-429f-a834-495daafa1be2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:46:23.058755 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-83112bf5-9de2-429f-a834-495daafa1be2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:46:23.060281 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1436/2859] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:46:22 2022] DELETE /identity/v3/projects/925a9005d7494b04923bdaa194731362 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:46:23.064356 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbeaae7-881c-4536-bc06-80dbfec5f294 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:46:23.065973 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2dbeaae7-881c-4536-bc06-80dbfec5f294 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:46:23.081487 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbeaae7-881c-4536-bc06-80dbfec5f294 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:46:23.136510 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26edca55-82e4-41e0-a2de-6fbc8086dbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:46:23.137370 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26edca55-82e4-41e0-a2de-6fbc8086dbec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:23.137656 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26edca55-82e4-41e0-a2de-6fbc8086dbec admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:23.137935 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26edca55-82e4-41e0-a2de-6fbc8086dbec admin admin] PATH_INFO: `/v3/users/850d4a3d03434a9dac9814f5b4a70db4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:23.139122 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26edca55-82e4-41e0-a2de-6fbc8086dbec admin admin] RBAC: Authorizing `identity:delete_user(user_id=850d4a3d03434a9dac9814f5b4a70db4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:46:23.141047 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26edca55-82e4-41e0-a2de-6fbc8086dbec admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:46:23.184523 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-26edca55-82e4-41e0-a2de-6fbc8086dbec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '850d4a3d03434a9dac9814f5b4a70db4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:46:23.191024 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-26edca55-82e4-41e0-a2de-6fbc8086dbec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '850d4a3d03434a9dac9814f5b4a70db4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:46:23.203390 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-26edca55-82e4-41e0-a2de-6fbc8086dbec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '850d4a3d03434a9dac9814f5b4a70db4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:46:23.204019 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-26edca55-82e4-41e0-a2de-6fbc8086dbec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '850d4a3d03434a9dac9814f5b4a70db4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:46:23.204641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-26edca55-82e4-41e0-a2de-6fbc8086dbec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '850d4a3d03434a9dac9814f5b4a70db4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:46:23.216516 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1424/2860] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:46:23 2022] DELETE /identity/v3/users/850d4a3d03434a9dac9814f5b4a70db4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:46:23.387775 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83112bf5-9de2-429f-a834-495daafa1be2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:46:23.389215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-83112bf5-9de2-429f-a834-495daafa1be2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:46:23.404173 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83112bf5-9de2-429f-a834-495daafa1be2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:46:23.423167 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:46:23.464206 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd608e11-e255-4d88-bf29-11d820e6e32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:46:23.465069 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd608e11-e255-4d88-bf29-11d820e6e32e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:23.465656 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd608e11-e255-4d88-bf29-11d820e6e32e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:23.465656 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd608e11-e255-4d88-bf29-11d820e6e32e admin admin] PATH_INFO: `/v3/projects/568410b0a79d457285548e6e78cbed6c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:23.466979 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd608e11-e255-4d88-bf29-11d820e6e32e admin admin] RBAC: Authorizing `identity:delete_project(project_id=568410b0a79d457285548e6e78cbed6c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:46:23.468750 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd608e11-e255-4d88-bf29-11d820e6e32e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:46:23.514797 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cd608e11-e255-4d88-bf29-11d820e6e32e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '568410b0a79d457285548e6e78cbed6c', 'request_id': 'req-cd608e11-e255-4d88-bf29-11d820e6e32e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:46:23.522632 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cd608e11-e255-4d88-bf29-11d820e6e32e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '568410b0a79d457285548e6e78cbed6c', 'request_id': 'req-cd608e11-e255-4d88-bf29-11d820e6e32e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:46:23.528952 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cd608e11-e255-4d88-bf29-11d820e6e32e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '568410b0a79d457285548e6e78cbed6c', 'request_id': 'req-cd608e11-e255-4d88-bf29-11d820e6e32e'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:46:23.539399 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cd608e11-e255-4d88-bf29-11d820e6e32e admin admin] The token cache is being invalidate because project 568410b0a79d457285548e6e78cbed6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:46:23.539692 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cd608e11-e255-4d88-bf29-11d820e6e32e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:46:23.540331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cd608e11-e255-4d88-bf29-11d820e6e32e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:46:23.541875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1437/2861] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:46:23 2022] DELETE /identity/v3/projects/568410b0a79d457285548e6e78cbed6c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:46:23.546884 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fce40-48ef-42b8-ba9e-bc38c6d90e56 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:23.547375 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fce40-48ef-42b8-ba9e-bc38c6d90e56 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:23.547486 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747fce40-48ef-42b8-ba9e-bc38c6d90e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:23.603600 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-747fce40-48ef-42b8-ba9e-bc38c6d90e56 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:46:23.604714 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-747fce40-48ef-42b8-ba9e-bc38c6d90e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:46:23.651611 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:46:23.670559 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1425/2862] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:46:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:46:23.678987 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e2774-50d3-4088-829c-2529e4293920 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:23.679278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e2774-50d3-4088-829c-2529e4293920 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:23.679556 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e2774-50d3-4088-829c-2529e4293920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:23.697537 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a99e2774-50d3-4088-829c-2529e4293920 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:46:23.698770 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a99e2774-50d3-4088-829c-2529e4293920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:46:23.706536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1438/2863] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:46:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:46:23.711064 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fce40-48ef-42b8-ba9e-bc38c6d90e56 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:46:23.720732 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747fce40-48ef-42b8-ba9e-bc38c6d90e56 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:46:23.725912 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90ae8b-2a40-438e-a95f-f9f6b4927a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:46:23.726594 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90ae8b-2a40-438e-a95f-f9f6b4927a5e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:23.726885 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90ae8b-2a40-438e-a95f-f9f6b4927a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:23.727168 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90ae8b-2a40-438e-a95f-f9f6b4927a5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:23.727898 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a90ae8b-2a40-438e-a95f-f9f6b4927a5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 29 12:46:23.728211 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a90ae8b-2a40-438e-a95f-f9f6b4927a5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:46:23.729816 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a90ae8b-2a40-438e-a95f-f9f6b4927a5e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:46:23.741416 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1426/2864] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jun 29 12:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 29 12:46:23.745678 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e2774-50d3-4088-829c-2529e4293920 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:46:23.754824 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e2774-50d3-4088-829c-2529e4293920 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:46:23.759470 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63da6ff-0a7c-4c99-a3eb-e8c0883f55ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:46:23.760118 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63da6ff-0a7c-4c99-a3eb-e8c0883f55ae admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:23.760407 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63da6ff-0a7c-4c99-a3eb-e8c0883f55ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:23.760691 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63da6ff-0a7c-4c99-a3eb-e8c0883f55ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:23.761381 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63da6ff-0a7c-4c99-a3eb-e8c0883f55ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:46:23.763307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63da6ff-0a7c-4c99-a3eb-e8c0883f55ae admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:46:23.789427 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1439/2865] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:46:23 2022] POST /identity/v3/projects => generated 379 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:46:23.793468 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90ae8b-2a40-438e-a95f-f9f6b4927a5e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:46:23.803616 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90ae8b-2a40-438e-a95f-f9f6b4927a5e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:46:23.858446 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d345f-c381-4a23-ab0a-21a95e9b31b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:46:23.859050 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d345f-c381-4a23-ab0a-21a95e9b31b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:23.859340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d345f-c381-4a23-ab0a-21a95e9b31b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:23.859623 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d345f-c381-4a23-ab0a-21a95e9b31b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:23.860308 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d345f-c381-4a23-ab0a-21a95e9b31b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:46:23.862230 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4d345f-c381-4a23-ab0a-21a95e9b31b4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:46:23.882822 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1427/2866] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:46:23 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:46:23.886904 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63da6ff-0a7c-4c99-a3eb-e8c0883f55ae admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:46:23.897311 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63da6ff-0a7c-4c99-a3eb-e8c0883f55ae admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:46:23.902567 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9bb21-70bd-49b4-ae1f-5b2a585d59fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:46:23.903230 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9bb21-70bd-49b4-ae1f-5b2a585d59fd admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:23.903519 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9bb21-70bd-49b4-ae1f-5b2a585d59fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:23.903802 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9bb21-70bd-49b4-ae1f-5b2a585d59fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:23.904527 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c9bb21-70bd-49b4-ae1f-5b2a585d59fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:46:23.906236 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c9bb21-70bd-49b4-ae1f-5b2a585d59fd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:46:23.916825 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1440/2867] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:46:23.920603 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d345f-c381-4a23-ab0a-21a95e9b31b4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:46:23.931027 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d345f-c381-4a23-ab0a-21a95e9b31b4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:46:23.935591 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009d56e-0a4a-4529-9e73-f6dd7b314bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:46:23.936263 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009d56e-0a4a-4529-9e73-f6dd7b314bd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:23.936534 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009d56e-0a4a-4529-9e73-f6dd7b314bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:23.936815 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009d56e-0a4a-4529-9e73-f6dd7b314bd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:23.937593 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009d56e-0a4a-4529-9e73-f6dd7b314bd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:46:23.939227 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009d56e-0a4a-4529-9e73-f6dd7b314bd3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:46:23.949512 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1428/2868] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:46:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:46:23.953407 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9bb21-70bd-49b4-ae1f-5b2a585d59fd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:46:23.964307 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9bb21-70bd-49b4-ae1f-5b2a585d59fd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:46:23.968923 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3c651-d556-4ead-8c16-48fcba225737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:46:23.969929 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3c651-d556-4ead-8c16-48fcba225737 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:23.970215 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3c651-d556-4ead-8c16-48fcba225737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:23.970536 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3c651-d556-4ead-8c16-48fcba225737 admin admin] PATH_INFO: `/v3/projects/5c3c5d6a04e74a23984b32760c5b1e87/users/b08abc69fc8e46cea2831101e8d5db87/roles/3bf034b395b44d88b3af67da01a6ed82` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:23.985147 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db3c651-d556-4ead-8c16-48fcba225737 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c3c5d6a04e74a23984b32760c5b1e87, user_id=b08abc69fc8e46cea2831101e8d5db87, role_id=3bf034b395b44d88b3af67da01a6ed82)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:46:23.987828 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db3c651-d556-4ead-8c16-48fcba225737 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:46:24.000895 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1441/2869] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:46:23 2022] PUT /identity/v3/projects/5c3c5d6a04e74a23984b32760c5b1e87/users/b08abc69fc8e46cea2831101e8d5db87/roles/3bf034b395b44d88b3af67da01a6ed82 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:46:24.006522 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab192b-4d89-4e59-b96d-7cb185a40d48 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:24.006929 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab192b-4d89-4e59-b96d-7cb185a40d48 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:24.007241 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab192b-4d89-4e59-b96d-7cb185a40d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:24.024954 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e6ab192b-4d89-4e59-b96d-7cb185a40d48 None None] MFA Rules not processed for user `96982c6fb7694d38b8d0a9278de02353`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:46:24.026153 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e6ab192b-4d89-4e59-b96d-7cb185a40d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:46:24.084758 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1429/2870] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:46:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:46:24.094272 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3c651-d556-4ead-8c16-48fcba225737 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:46:24.096028 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7db3c651-d556-4ead-8c16-48fcba225737 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:46:24.152487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:46:24.179034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3c651-d556-4ead-8c16-48fcba225737 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:46:24.182839 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d272cc-1345-4d3b-83c7-e7627dc2aec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:46:24.183509 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d272cc-1345-4d3b-83c7-e7627dc2aec3 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:24.183801 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d272cc-1345-4d3b-83c7-e7627dc2aec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:24.184084 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d272cc-1345-4d3b-83c7-e7627dc2aec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:24.193860 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d272cc-1345-4d3b-83c7-e7627dc2aec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:46:24.195331 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d272cc-1345-4d3b-83c7-e7627dc2aec3 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:46:24.204718 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1442/2871] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:46:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:46:28.202314 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d237b-8332-46f7-ba6b-4462ef3c1b81 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:28.202838 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d237b-8332-46f7-ba6b-4462ef3c1b81 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:28.203095 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2d237b-8332-46f7-ba6b-4462ef3c1b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:28.233961 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cf2d237b-8332-46f7-ba6b-4462ef3c1b81 None None] MFA Rules not processed for user `b08abc69fc8e46cea2831101e8d5db87`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:46:28.235217 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cf2d237b-8332-46f7-ba6b-4462ef3c1b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:46:28.280541 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:46:28.307428 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1430/2872] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:46:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:46:28.311963 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d272cc-1345-4d3b-83c7-e7627dc2aec3 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:46:28.323289 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d272cc-1345-4d3b-83c7-e7627dc2aec3 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:46:28.328071 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991b570-ff67-421f-8b95-1cc12b8512d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:46:28.328740 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991b570-ff67-421f-8b95-1cc12b8512d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:28.329026 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991b570-ff67-421f-8b95-1cc12b8512d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:28.329303 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991b570-ff67-421f-8b95-1cc12b8512d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:28.330056 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991b570-ff67-421f-8b95-1cc12b8512d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:46:28.332004 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991b570-ff67-421f-8b95-1cc12b8512d5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:46:28.361550 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1443/2873] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jun 29 12:46:28 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:46:28.367422 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2d237b-8332-46f7-ba6b-4462ef3c1b81 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:46:28.378796 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2d237b-8332-46f7-ba6b-4462ef3c1b81 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:46:28.447488 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841178a2-e377-4f4e-8eab-245639bfdf0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:46:28.448175 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841178a2-e377-4f4e-8eab-245639bfdf0a admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:28.448464 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841178a2-e377-4f4e-8eab-245639bfdf0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:28.448743 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841178a2-e377-4f4e-8eab-245639bfdf0a admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:28.449539 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841178a2-e377-4f4e-8eab-245639bfdf0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:46:28.451538 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841178a2-e377-4f4e-8eab-245639bfdf0a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:46:28.473649 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1431/2874] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jun 29 12:46:28 2022] POST /identity/v3/users => generated 296 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 29 12:46:28.478480 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991b570-ff67-421f-8b95-1cc12b8512d5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:46:28.489615 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991b570-ff67-421f-8b95-1cc12b8512d5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:46:28.495920 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd666dee-eb7a-4086-81c5-7788782e13a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:46:28.496690 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd666dee-eb7a-4086-81c5-7788782e13a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:28.497048 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd666dee-eb7a-4086-81c5-7788782e13a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:28.497399 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd666dee-eb7a-4086-81c5-7788782e13a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:28.498203 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd666dee-eb7a-4086-81c5-7788782e13a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:46:28.499972 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd666dee-eb7a-4086-81c5-7788782e13a7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:46:28.511775 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1444/2875] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 29 12:46:28.515895 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841178a2-e377-4f4e-8eab-245639bfdf0a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:46:28.527232 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841178a2-e377-4f4e-8eab-245639bfdf0a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:46:28.532089 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d56bd-92e1-4fa2-aab5-7d7b2f97b52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:46:28.532998 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d56bd-92e1-4fa2-aab5-7d7b2f97b52a admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:28.533289 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d56bd-92e1-4fa2-aab5-7d7b2f97b52a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:28.533571 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d56bd-92e1-4fa2-aab5-7d7b2f97b52a admin admin] PATH_INFO: `/v3/projects/5514caf199d2444eb3041686b83af42e/users/ae1c24da66154c0b9d0267e32cb90ff3/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:28.551502 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d56bd-92e1-4fa2-aab5-7d7b2f97b52a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5514caf199d2444eb3041686b83af42e, user_id=ae1c24da66154c0b9d0267e32cb90ff3, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:46:28.554641 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d56bd-92e1-4fa2-aab5-7d7b2f97b52a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:46:28.569699 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1432/2876] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:46:28 2022] PUT /identity/v3/projects/5514caf199d2444eb3041686b83af42e/users/ae1c24da66154c0b9d0267e32cb90ff3/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:46:28.573675 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd666dee-eb7a-4086-81c5-7788782e13a7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:46:28.586332 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd666dee-eb7a-4086-81c5-7788782e13a7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:46:28.649342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65655c6e-c8f5-4266-b112-db788cdd1718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:46:28.650161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65655c6e-c8f5-4266-b112-db788cdd1718 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:28.650490 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65655c6e-c8f5-4266-b112-db788cdd1718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:28.650777 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65655c6e-c8f5-4266-b112-db788cdd1718 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:28.651508 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65655c6e-c8f5-4266-b112-db788cdd1718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:46:28.653108 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65655c6e-c8f5-4266-b112-db788cdd1718 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:46:28.663243 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1445/2877] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jun 29 12:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 29 12:46:28.667483 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d56bd-92e1-4fa2-aab5-7d7b2f97b52a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:46:28.677996 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d56bd-92e1-4fa2-aab5-7d7b2f97b52a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:46:28.682541 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b07beb-4c73-45ca-aed6-c9df326a0e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:46:28.683445 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b07beb-4c73-45ca-aed6-c9df326a0e31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:28.683728 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b07beb-4c73-45ca-aed6-c9df326a0e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:28.684004 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b07beb-4c73-45ca-aed6-c9df326a0e31 admin admin] PATH_INFO: `/v3/projects/5514caf199d2444eb3041686b83af42e/users/ae1c24da66154c0b9d0267e32cb90ff3/roles/3a7f4bf556344407a6d9540c89d13641` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:28.685947 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b07beb-4c73-45ca-aed6-c9df326a0e31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5514caf199d2444eb3041686b83af42e, user_id=ae1c24da66154c0b9d0267e32cb90ff3, role_id=3a7f4bf556344407a6d9540c89d13641)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:46:28.688572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b07beb-4c73-45ca-aed6-c9df326a0e31 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:46:28.699165 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1433/2878] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jun 29 12:46:28 2022] PUT /identity/v3/projects/5514caf199d2444eb3041686b83af42e/users/ae1c24da66154c0b9d0267e32cb90ff3/roles/3a7f4bf556344407a6d9540c89d13641 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:46:33.367898 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eed563-244c-4e8d-9921-8ead3230559e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:33.368429 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eed563-244c-4e8d-9921-8ead3230559e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:33.368710 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eed563-244c-4e8d-9921-8ead3230559e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:33.399791 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-12eed563-244c-4e8d-9921-8ead3230559e None None] MFA Rules not processed for user `ae1c24da66154c0b9d0267e32cb90ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:46:33.401129 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-12eed563-244c-4e8d-9921-8ead3230559e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:46:33.482953 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1446/2879] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:46:33 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:46:33.498667 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b07beb-4c73-45ca-aed6-c9df326a0e31 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:46:33.501052 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-74b07beb-4c73-45ca-aed6-c9df326a0e31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:46:33.611198 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b07beb-4c73-45ca-aed6-c9df326a0e31 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:46:33.619237 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb80fe-fe27-499a-ba42-91b99937646f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:46:33.619916 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb80fe-fe27-499a-ba42-91b99937646f service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:33.620322 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb80fe-fe27-499a-ba42-91b99937646f service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:33.620696 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb80fe-fe27-499a-ba42-91b99937646f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:33.635629 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:46:33.691904 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb80fe-fe27-499a-ba42-91b99937646f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:46:33.692488 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb80fe-fe27-499a-ba42-91b99937646f service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:46:33.703092 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1434/2880] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:46:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:46:43.479555 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:46:43.698272 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=22 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:46:49.959988 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4dd9b4-cd5b-4b74-b1fa-7e38a25853ea None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:49.960487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4dd9b4-cd5b-4b74-b1fa-7e38a25853ea None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:49.961193 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4dd9b4-cd5b-4b74-b1fa-7e38a25853ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:49.979199 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9c4dd9b4-cd5b-4b74-b1fa-7e38a25853ea None None] MFA Rules not processed for user `b08abc69fc8e46cea2831101e8d5db87`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:46:49.980390 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9c4dd9b4-cd5b-4b74-b1fa-7e38a25853ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:46:49.987157 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1447/2881] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:46:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 29 12:46:50.300301 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb80fe-fe27-499a-ba42-91b99937646f service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:46:50.354592 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb80fe-fe27-499a-ba42-91b99937646f service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:46:50.358202 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f362f9f-9894-4912-a188-fd7b0d0a0ab7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:46:50.358891 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f362f9f-9894-4912-a188-fd7b0d0a0ab7 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:50.359170 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f362f9f-9894-4912-a188-fd7b0d0a0ab7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:50.359444 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f362f9f-9894-4912-a188-fd7b0d0a0ab7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:50.361290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9f362f9f-9894-4912-a188-fd7b0d0a0ab7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:46:50.372460 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f362f9f-9894-4912-a188-fd7b0d0a0ab7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:46:50.373808 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f362f9f-9894-4912-a188-fd7b0d0a0ab7 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:46:50.381832 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1435/2882] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:46:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:46:52.270390 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4dd9b4-cd5b-4b74-b1fa-7e38a25853ea None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:46:52.271812 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9c4dd9b4-cd5b-4b74-b1fa-7e38a25853ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:46:52.363845 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4dd9b4-cd5b-4b74-b1fa-7e38a25853ea None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:46:52.369623 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18371b-d4eb-4728-b501-4aeba9d43442 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98135cacb9d4f22846ad33500b9bd1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:46:52.370279 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18371b-d4eb-4728-b501-4aeba9d43442 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:46:52.370612 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18371b-d4eb-4728-b501-4aeba9d43442 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:46:52.370892 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18371b-d4eb-4728-b501-4aeba9d43442 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:46:52.372662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bf18371b-d4eb-4728-b501-4aeba9d43442 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:46:52.384784 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf18371b-d4eb-4728-b501-4aeba9d43442 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:46:52.386351 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf18371b-d4eb-4728-b501-4aeba9d43442 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:46:52.396222 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1448/2883] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:46:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:47:00.378244 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=26 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:47:02.391917 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=44 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:47:06.069191 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f362f9f-9894-4912-a188-fd7b0d0a0ab7 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:47:06.078331 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f362f9f-9894-4912-a188-fd7b0d0a0ab7 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:47:06.081943 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e0f702-efa7-4f8d-b809-2f6e5d6af4ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:47:06.082601 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0f702-efa7-4f8d-b809-2f6e5d6af4ba service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:47:06.082885 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0f702-efa7-4f8d-b809-2f6e5d6af4ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:47:06.083157 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e0f702-efa7-4f8d-b809-2f6e5d6af4ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:47:06.092923 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e0f702-efa7-4f8d-b809-2f6e5d6af4ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:47:06.094340 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e0f702-efa7-4f8d-b809-2f6e5d6af4ba service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:47:06.104284 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1436/2884] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:47:06 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:47:16.099515 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:49:03.019188 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18371b-d4eb-4728-b501-4aeba9d43442 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:49:03.020655 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bf18371b-d4eb-4728-b501-4aeba9d43442 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:49:03.033348 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18371b-d4eb-4728-b501-4aeba9d43442 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:49:03.036907 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd02b1-172b-4327-9bf5-b8f8feaee8b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:49:03.037540 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd02b1-172b-4327-9bf5-b8f8feaee8b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:49:03.037825 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd02b1-172b-4327-9bf5-b8f8feaee8b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:49:03.038104 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd02b1-172b-4327-9bf5-b8f8feaee8b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:49:03.040487 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e5fd02b1-172b-4327-9bf5-b8f8feaee8b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:49:03.054241 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd02b1-172b-4327-9bf5-b8f8feaee8b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:49:03.055718 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd02b1-172b-4327-9bf5-b8f8feaee8b6 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:49:03.066688 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1449/2885] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:49:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:49:04.239918 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e0f702-efa7-4f8d-b809-2f6e5d6af4ba service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:49:04.248849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e0f702-efa7-4f8d-b809-2f6e5d6af4ba service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:49:04.252458 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5d32f-2fb6-4161-8a35-0ad60502039e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:49:04.253087 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5d32f-2fb6-4161-8a35-0ad60502039e service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:49:04.253369 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5d32f-2fb6-4161-8a35-0ad60502039e service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:49:04.253648 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5d32f-2fb6-4161-8a35-0ad60502039e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:49:04.255378 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8ec5d32f-2fb6-4161-8a35-0ad60502039e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:49:04.268345 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5d32f-2fb6-4161-8a35-0ad60502039e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:49:04.269761 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec5d32f-2fb6-4161-8a35-0ad60502039e service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:49:04.281592 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1437/2886] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:49:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:49:06.555871 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c73d98-fd5e-49d0-8fd6-b25c38fc2b65 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:49:06.556342 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c73d98-fd5e-49d0-8fd6-b25c38fc2b65 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:49:06.556455 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c73d98-fd5e-49d0-8fd6-b25c38fc2b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:49:06.573532 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b5c73d98-fd5e-49d0-8fd6-b25c38fc2b65 None None] MFA Rules not processed for user `b08abc69fc8e46cea2831101e8d5db87`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:49:06.574751 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b5c73d98-fd5e-49d0-8fd6-b25c38fc2b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:49:06.581434 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1450/2887] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:49:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:49:14.276782 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:49:16.568237 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=9 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:49:37.145955 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5d32f-2fb6-4161-8a35-0ad60502039e service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:49:37.156290 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5d32f-2fb6-4161-8a35-0ad60502039e service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:49:37.159873 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed055a3d-9bcf-4d62-ae78-3136b959732b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:49:37.160496 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed055a3d-9bcf-4d62-ae78-3136b959732b service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:49:37.160779 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed055a3d-9bcf-4d62-ae78-3136b959732b service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:49:37.161056 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed055a3d-9bcf-4d62-ae78-3136b959732b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:49:37.162818 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ed055a3d-9bcf-4d62-ae78-3136b959732b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:49:37.172233 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed055a3d-9bcf-4d62-ae78-3136b959732b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:49:37.173660 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed055a3d-9bcf-4d62-ae78-3136b959732b service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:49:37.181631 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1438/2888] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:49:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:49:47.178020 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:50:55.133772 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf0432-5575-4bda-a4ef-e1200012fb2f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:50:55.134278 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf0432-5575-4bda-a4ef-e1200012fb2f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:50:55.134388 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf0432-5575-4bda-a4ef-e1200012fb2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:50:55.152407 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a8cf0432-5575-4bda-a4ef-e1200012fb2f None None] MFA Rules not processed for user `b08abc69fc8e46cea2831101e8d5db87`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 29 12:50:55.153599 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a8cf0432-5575-4bda-a4ef-e1200012fb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:50:55.161095 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1451/2889] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jun 29 12:50:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 29 12:50:56.924051 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed055a3d-9bcf-4d62-ae78-3136b959732b service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:50:56.936849 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed055a3d-9bcf-4d62-ae78-3136b959732b service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:50:56.941991 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c01921-ffbd-4cbc-922c-4aaefae872fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98135cacb9d4f22846ad33500b9bd1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:50:56.942767 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c01921-ffbd-4cbc-922c-4aaefae872fe service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:50:56.943052 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c01921-ffbd-4cbc-922c-4aaefae872fe service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:50:56.943330 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c01921-ffbd-4cbc-922c-4aaefae872fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:50:56.945057 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-50c01921-ffbd-4cbc-922c-4aaefae872fe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:50:56.956258 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c01921-ffbd-4cbc-922c-4aaefae872fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:50:56.957842 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c01921-ffbd-4cbc-922c-4aaefae872fe service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:50:56.967572 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1439/2890] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:50:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 29 12:51:02.207522 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf0432-5575-4bda-a4ef-e1200012fb2f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:51:02.231280 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf0432-5575-4bda-a4ef-e1200012fb2f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:51:02.239194 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc34197-dc4c-4a5e-9ff1-fb033414af1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:51:02.240409 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc34197-dc4c-4a5e-9ff1-fb033414af1d service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:51:02.240862 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc34197-dc4c-4a5e-9ff1-fb033414af1d service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:51:02.241369 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc34197-dc4c-4a5e-9ff1-fb033414af1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:51:02.244170 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0fc34197-dc4c-4a5e-9ff1-fb033414af1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:51:02.259292 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc34197-dc4c-4a5e-9ff1-fb033414af1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:51:02.261395 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc34197-dc4c-4a5e-9ff1-fb033414af1d service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:51:02.274130 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1452/2891] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:51:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:51:06.963345 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:51:12.269093 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=9 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:51:12.851771 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c01921-ffbd-4cbc-922c-4aaefae872fe service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:51:12.866800 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c01921-ffbd-4cbc-922c-4aaefae872fe service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:51:12.872335 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10cc2e-d134-456f-9bfc-a546af2d0efd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3ac7391a3c41499d94e5f97478797313', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:51:12.873518 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10cc2e-d134-456f-9bfc-a546af2d0efd service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:51:12.873518 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10cc2e-d134-456f-9bfc-a546af2d0efd service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:51:12.873883 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10cc2e-d134-456f-9bfc-a546af2d0efd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:51:12.875797 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1f10cc2e-d134-456f-9bfc-a546af2d0efd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:51:12.889343 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f10cc2e-d134-456f-9bfc-a546af2d0efd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:51:12.891057 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f10cc2e-d134-456f-9bfc-a546af2d0efd service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:51:12.903533 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1440/2892] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:51:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:51:22.897749 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:52:37.418840 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc34197-dc4c-4a5e-9ff1-fb033414af1d service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:52:37.420238 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0fc34197-dc4c-4a5e-9ff1-fb033414af1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:52:37.435262 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc34197-dc4c-4a5e-9ff1-fb033414af1d service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:52:37.440250 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f87cc-2d3c-4b84-8eed-667f25178aa8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98135cacb9d4f22846ad33500b9bd1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:52:37.440892 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f87cc-2d3c-4b84-8eed-667f25178aa8 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:52:37.441183 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f87cc-2d3c-4b84-8eed-667f25178aa8 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:52:37.441466 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f87cc-2d3c-4b84-8eed-667f25178aa8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:52:37.450825 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f87cc-2d3c-4b84-8eed-667f25178aa8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:52:37.452396 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f87cc-2d3c-4b84-8eed-667f25178aa8 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:52:37.462463 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1453/2893] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:52:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:52:46.426676 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10cc2e-d134-456f-9bfc-a546af2d0efd service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:52:46.436286 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10cc2e-d134-456f-9bfc-a546af2d0efd service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:52:46.439818 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4cf2a1-fa22-44a1-8b93-15dc15c18cd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:52:46.440483 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cf2a1-fa22-44a1-8b93-15dc15c18cd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:52:46.440768 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cf2a1-fa22-44a1-8b93-15dc15c18cd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:52:46.441044 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cf2a1-fa22-44a1-8b93-15dc15c18cd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:52:46.449251 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4cf2a1-fa22-44a1-8b93-15dc15c18cd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:52:46.450715 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4cf2a1-fa22-44a1-8b93-15dc15c18cd3 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:52:46.459422 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1441/2894] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:52:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:52:47.458034 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:52:56.455485 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:52:59.755992 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f87cc-2d3c-4b84-8eed-667f25178aa8 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:52:59.765877 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f87cc-2d3c-4b84-8eed-667f25178aa8 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:52:59.769644 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fed25-2672-485e-a03a-e50b85caa049 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce922cdbf2d34f10b6e49631f0c6d770', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:52:59.770295 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fed25-2672-485e-a03a-e50b85caa049 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:52:59.770606 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fed25-2672-485e-a03a-e50b85caa049 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:52:59.770887 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fed25-2672-485e-a03a-e50b85caa049 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:52:59.830385 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7fed25-2672-485e-a03a-e50b85caa049 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:52:59.831811 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7fed25-2672-485e-a03a-e50b85caa049 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:52:59.841292 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1454/2895] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:52:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:53:02.904830 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4cf2a1-fa22-44a1-8b93-15dc15c18cd3 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:53:02.918619 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4cf2a1-fa22-44a1-8b93-15dc15c18cd3 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:53:02.923821 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac81257-a903-4006-a183-7b370c028866 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98135cacb9d4f22846ad33500b9bd1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:53:02.924598 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac81257-a903-4006-a183-7b370c028866 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:53:02.924995 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac81257-a903-4006-a183-7b370c028866 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:53:02.925378 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac81257-a903-4006-a183-7b370c028866 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:53:02.927480 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0ac81257-a903-4006-a183-7b370c028866 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:53:02.946828 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac81257-a903-4006-a183-7b370c028866 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:53:02.948466 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac81257-a903-4006-a183-7b370c028866 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:53:02.969538 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1442/2896] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:53:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:53:06.121547 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fed25-2672-485e-a03a-e50b85caa049 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:53:06.140856 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7fed25-2672-485e-a03a-e50b85caa049 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:53:06.145389 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88267ff4-6d91-4bc4-99d8-3b54dbc714b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:53:06.146266 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88267ff4-6d91-4bc4-99d8-3b54dbc714b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:53:06.146581 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88267ff4-6d91-4bc4-99d8-3b54dbc714b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:53:06.146862 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88267ff4-6d91-4bc4-99d8-3b54dbc714b6 admin admin] PATH_INFO: `/v3/users/b08abc69fc8e46cea2831101e8d5db87` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:53:06.148032 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88267ff4-6d91-4bc4-99d8-3b54dbc714b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b08abc69fc8e46cea2831101e8d5db87)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:53:06.150164 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88267ff4-6d91-4bc4-99d8-3b54dbc714b6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:53:06.196374 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-88267ff4-6d91-4bc4-99d8-3b54dbc714b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b08abc69fc8e46cea2831101e8d5db87'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:53:06.197097 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-88267ff4-6d91-4bc4-99d8-3b54dbc714b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b08abc69fc8e46cea2831101e8d5db87'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:53:06.208604 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-88267ff4-6d91-4bc4-99d8-3b54dbc714b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b08abc69fc8e46cea2831101e8d5db87'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:53:06.209161 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-88267ff4-6d91-4bc4-99d8-3b54dbc714b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b08abc69fc8e46cea2831101e8d5db87'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:53:06.216479 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-88267ff4-6d91-4bc4-99d8-3b54dbc714b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b08abc69fc8e46cea2831101e8d5db87'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:53:06.230149 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1455/2897] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:53:06 2022] DELETE /identity/v3/users/b08abc69fc8e46cea2831101e8d5db87 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:53:06.401774 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac81257-a903-4006-a183-7b370c028866 service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:53:06.412792 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac81257-a903-4006-a183-7b370c028866 service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:53:06.470367 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792e12c-498b-47cf-8354-654cd18345f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:53:06.471264 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792e12c-498b-47cf-8354-654cd18345f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:53:06.471555 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792e12c-498b-47cf-8354-654cd18345f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:53:06.471837 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792e12c-498b-47cf-8354-654cd18345f2 admin admin] PATH_INFO: `/v3/projects/5c3c5d6a04e74a23984b32760c5b1e87` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:53:06.473018 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792e12c-498b-47cf-8354-654cd18345f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c3c5d6a04e74a23984b32760c5b1e87)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:53:06.475010 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792e12c-498b-47cf-8354-654cd18345f2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:53:06.521296 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e792e12c-498b-47cf-8354-654cd18345f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c3c5d6a04e74a23984b32760c5b1e87', 'request_id': 'req-e792e12c-498b-47cf-8354-654cd18345f2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:53:06.527324 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e792e12c-498b-47cf-8354-654cd18345f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c3c5d6a04e74a23984b32760c5b1e87', 'request_id': 'req-e792e12c-498b-47cf-8354-654cd18345f2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:53:06.533239 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e792e12c-498b-47cf-8354-654cd18345f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c3c5d6a04e74a23984b32760c5b1e87', 'request_id': 'req-e792e12c-498b-47cf-8354-654cd18345f2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:53:06.545774 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e792e12c-498b-47cf-8354-654cd18345f2 admin admin] The token cache is being invalidate because project 5c3c5d6a04e74a23984b32760c5b1e87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:53:06.546191 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e792e12c-498b-47cf-8354-654cd18345f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:53:06.546825 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e792e12c-498b-47cf-8354-654cd18345f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:53:06.548334 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1443/2898] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:53:06 2022] DELETE /identity/v3/projects/5c3c5d6a04e74a23984b32760c5b1e87 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:53:06.552444 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88267ff4-6d91-4bc4-99d8-3b54dbc714b6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:53:06.553853 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-88267ff4-6d91-4bc4-99d8-3b54dbc714b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:53:06.567299 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88267ff4-6d91-4bc4-99d8-3b54dbc714b6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:53:06.623435 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc9d466-861d-44f8-9d20-d85e3a10f9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:53:06.624303 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9d466-861d-44f8-9d20-d85e3a10f9ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:53:06.624588 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9d466-861d-44f8-9d20-d85e3a10f9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:53:06.624863 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc9d466-861d-44f8-9d20-d85e3a10f9ad admin admin] PATH_INFO: `/v3/users/ae1c24da66154c0b9d0267e32cb90ff3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:53:06.626037 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc9d466-861d-44f8-9d20-d85e3a10f9ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae1c24da66154c0b9d0267e32cb90ff3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:53:06.627955 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc9d466-861d-44f8-9d20-d85e3a10f9ad admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:53:06.633121 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:53:06.673779 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8cc9d466-861d-44f8-9d20-d85e3a10f9ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae1c24da66154c0b9d0267e32cb90ff3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:53:06.674353 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8cc9d466-861d-44f8-9d20-d85e3a10f9ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae1c24da66154c0b9d0267e32cb90ff3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:53:06.686089 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8cc9d466-861d-44f8-9d20-d85e3a10f9ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae1c24da66154c0b9d0267e32cb90ff3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:53:06.686733 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8cc9d466-861d-44f8-9d20-d85e3a10f9ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae1c24da66154c0b9d0267e32cb90ff3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:53:06.693457 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8cc9d466-861d-44f8-9d20-d85e3a10f9ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae1c24da66154c0b9d0267e32cb90ff3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:53:06.706089 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1456/2899] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jun 29 12:53:06 2022] DELETE /identity/v3/users/ae1c24da66154c0b9d0267e32cb90ff3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:53:06.888454 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792e12c-498b-47cf-8354-654cd18345f2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:53:06.890001 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e792e12c-498b-47cf-8354-654cd18345f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:53:06.904147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792e12c-498b-47cf-8354-654cd18345f2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:53:06.967918 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454c83a-eff3-4f5c-8937-1e7e392390a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96982c6fb7694d38b8d0a9278de02353', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a19795d69941402bbd95f8c9172fb339', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:53:06.968781 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454c83a-eff3-4f5c-8937-1e7e392390a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:53:06.969235 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454c83a-eff3-4f5c-8937-1e7e392390a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:53:06.969380 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454c83a-eff3-4f5c-8937-1e7e392390a9 admin admin] PATH_INFO: `/v3/projects/5514caf199d2444eb3041686b83af42e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:53:06.970544 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b454c83a-eff3-4f5c-8937-1e7e392390a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5514caf199d2444eb3041686b83af42e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:53:06.972481 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b454c83a-eff3-4f5c-8937-1e7e392390a9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:53:06.994053 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:53:07.018999 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b454c83a-eff3-4f5c-8937-1e7e392390a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5514caf199d2444eb3041686b83af42e', 'request_id': 'req-b454c83a-eff3-4f5c-8937-1e7e392390a9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:53:07.025413 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b454c83a-eff3-4f5c-8937-1e7e392390a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5514caf199d2444eb3041686b83af42e', 'request_id': 'req-b454c83a-eff3-4f5c-8937-1e7e392390a9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:53:07.031147 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b454c83a-eff3-4f5c-8937-1e7e392390a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5514caf199d2444eb3041686b83af42e', 'request_id': 'req-b454c83a-eff3-4f5c-8937-1e7e392390a9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:53:07.043993 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b454c83a-eff3-4f5c-8937-1e7e392390a9 admin admin] The token cache is being invalidate because project 5514caf199d2444eb3041686b83af42e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 29 12:53:07.044280 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b454c83a-eff3-4f5c-8937-1e7e392390a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:53:07.044895 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b454c83a-eff3-4f5c-8937-1e7e392390a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 29 12:53:07.046521 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1444/2900] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jun 29 12:53:06 2022] DELETE /identity/v3/projects/5514caf199d2444eb3041686b83af42e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 29 12:53:16.702672 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:53:17.040740 dv-cinder-812685-12-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 29 12:57:40.362875 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc9d466-861d-44f8-9d20-d85e3a10f9ad admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 29 12:57:40.364765 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8cc9d466-861d-44f8-9d20-d85e3a10f9ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:57:40.502910 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc9d466-861d-44f8-9d20-d85e3a10f9ad admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 29 12:57:40.561241 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf8f93-2555-4e41-ada7-99ec84feeff1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c98135cacb9d4f22846ad33500b9bd1f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f043ade9e4f748d4adc010ffb0a47a46', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 29 12:57:40.561888 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf8f93-2555-4e41-ada7-99ec84feeff1 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 29 12:57:40.562180 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf8f93-2555-4e41-ada7-99ec84feeff1 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 29 12:57:40.562483 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf8f93-2555-4e41-ada7-99ec84feeff1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 29 12:57:40.564228 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-03cf8f93-2555-4e41-ada7-99ec84feeff1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 29 12:57:40.575801 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cf8f93-2555-4e41-ada7-99ec84feeff1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 29 12:57:40.577354 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cf8f93-2555-4e41-ada7-99ec84feeff1 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 29 12:57:40.587662 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1457/2901] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jun 29 12:57:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 29 12:57:50.583523 dv-cinder-812685-12-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}